2023-09-30T00:49:57.587 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-30T00:49:57.588 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821 --owner scheduled_yuriw@teuthology --verbose --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache} -- /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821/orig.config.yaml 2023-09-30T00:49:57.633 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T00:49:57.892 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821 branch: wip-yuri4-testing-2023-09-29-0928-quincy description: rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache} email: yweinste@redhat.com first_in_suite: false job_id: '7406821' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri4-testing-2023-09-29-0928-quincy ceph: conf: global: mon election default strategy: 1 ms bind msgr2: false ms type: async osd debug inject dispatch delay duration: 0.1 osd debug inject dispatch delay probability: 0.1 osd_async_recovery_min_cost: 1 osd_object_clean_region_max_num_intervals: 1000 osd_pool_default_min_size: 2 osd_pool_default_size: 2 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd inject bad map crc probability: 0.1 osd max backfills: 3 osd max markdown count: 1000 osd max scrubs: 2 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(REQUEST_STUCK\) - failed to encode map - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - must scrub before tier agent can activate mon_bind_msgr2: false sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2023-09-29-0928-quincy sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 9810 sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri4-testing-2023-09-29-0928-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: conf: osd: debug monc: 20 - exec: mon.a: - while ! ceph balancer status ; do sleep 1 ; done - ceph balancer mode crush-compat - ceph balancer on - thrashosds: aggressive_pg_num_changes: false chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache min_read_recency_for_promote 0 - sudo ceph osd pool set cache min_write_recency_for_promote 0 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 write: 100 ops: 4000 pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-29_19:38:56 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-30T00:49:57.893 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa; will attempt to use it 2023-09-30T00:49:57.894 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks 2023-09-30T00:49:57.894 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-30T00:49:57.896 INFO:teuthology.task.internal:Checking packages... 2023-09-30T00:49:57.919 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '93dce5e17918f8aca65993a7a2dc40233a47b481' 2023-09-30T00:49:57.919 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T00:49:57.919 INFO:teuthology.packaging:ref: None 2023-09-30T00:49:57.919 INFO:teuthology.packaging:tag: None 2023-09-30T00:49:57.919 INFO:teuthology.packaging:branch: wip-yuri4-testing-2023-09-29-0928-quincy 2023-09-30T00:49:57.919 INFO:teuthology.packaging:sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-30T00:49:57.920 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri4-testing-2023-09-29-0928-quincy 2023-09-30T00:49:58.139 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:49:58.140 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-30T00:49:58.146 INFO:teuthology.task.internal:no buildpackages task found 2023-09-30T00:49:58.146 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-30T00:49:58.163 INFO:teuthology.task.internal:Saving configuration 2023-09-30T00:49:58.180 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-30T00:49:58.185 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T00:49:58.210 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 00:44:10.785945', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-09-30T00:49:58.231 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 00:44:10.783937', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-09-30T00:49:58.231 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-30T00:49:58.239 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-09-30T00:49:58.239 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-09-30T00:49:58.240 INFO:teuthology.run_tasks:Running task console_log... 2023-09-30T00:49:58.308 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f16fbacc0d0>, signals=[15]) 2023-09-30T00:49:58.308 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-30T00:49:58.314 INFO:teuthology.task.internal:Opening connections... 2023-09-30T00:49:58.314 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2023-09-30T00:49:58.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:49:58.365 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2023-09-30T00:49:58.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:49:58.415 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-30T00:49:58.420 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2023-09-30T00:49:58.447 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2023-09-30T00:49:58.447 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2023-09-30T00:49:58.495 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="Ubuntu" 2023-09-30T00:49:58.495 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-09-30T00:49:58.495 INFO:teuthology.orchestra.run.smithi037.stdout:ID=ubuntu 2023-09-30T00:49:58.495 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE=debian 2023-09-30T00:49:58.495 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-09-30T00:49:58.495 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="20.04" 2023-09-30T00:49:58.495 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-30T00:49:58.495 INFO:teuthology.orchestra.run.smithi037.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-30T00:49:58.496 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-30T00:49:58.496 INFO:teuthology.orchestra.run.smithi037.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-30T00:49:58.496 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_CODENAME=focal 2023-09-30T00:49:58.496 INFO:teuthology.orchestra.run.smithi037.stdout:UBUNTU_CODENAME=focal 2023-09-30T00:49:58.496 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2023-09-30T00:49:58.516 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2023-09-30T00:49:58.522 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2023-09-30T00:49:58.523 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2023-09-30T00:49:58.573 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="Ubuntu" 2023-09-30T00:49:58.573 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-09-30T00:49:58.573 INFO:teuthology.orchestra.run.smithi050.stdout:ID=ubuntu 2023-09-30T00:49:58.573 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE=debian 2023-09-30T00:49:58.573 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-09-30T00:49:58.573 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="20.04" 2023-09-30T00:49:58.573 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-30T00:49:58.573 INFO:teuthology.orchestra.run.smithi050.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-30T00:49:58.573 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-30T00:49:58.573 INFO:teuthology.orchestra.run.smithi050.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-30T00:49:58.574 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_CODENAME=focal 2023-09-30T00:49:58.574 INFO:teuthology.orchestra.run.smithi050.stdout:UBUNTU_CODENAME=focal 2023-09-30T00:49:58.574 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2023-09-30T00:49:58.590 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-30T00:49:58.599 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-30T00:49:58.603 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-30T00:49:58.604 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2023-09-30T00:49:58.606 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2023-09-30T00:49:58.611 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-30T00:49:58.617 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-30T00:49:58.617 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2023-09-30T00:49:58.655 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2023-09-30T00:49:58.674 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-30T00:49:58.729 INFO:teuthology.run_tasks:Running task kernel... 2023-09-30T00:49:58.742 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-09-30T00:49:58.743 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-09-30T00:49:58.743 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T00:49:58.743 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T00:49:58.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:49:58.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:49:58.751 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2023-09-30T00:49:58.751 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2023-09-30T00:49:58.757 INFO:teuthology.orchestra.run.smithi050.stdout:5.4.0-124-generic 2023-09-30T00:49:58.757 INFO:teuthology.task.kernel:Running kernel on smithi050: 5.4.0-124-generic 2023-09-30T00:49:58.757 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-get clean 2023-09-30T00:49:58.758 INFO:teuthology.orchestra.run.smithi037.stdout:5.4.0-124-generic 2023-09-30T00:49:58.758 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.4.0-124-generic 2023-09-30T00:49:58.759 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-get clean 2023-09-30T00:49:58.903 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-get update 2023-09-30T00:49:58.996 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-get update 2023-09-30T00:49:59.059 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:49:59.062 INFO:teuthology.orchestra.run.smithi050.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-09-30T00:49:59.062 INFO:teuthology.orchestra.run.smithi050.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-09-30T00:49:59.131 INFO:teuthology.orchestra.run.smithi050.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-09-30T00:49:59.280 INFO:teuthology.orchestra.run.smithi037.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-09-30T00:49:59.392 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:49:59.454 INFO:teuthology.orchestra.run.smithi050.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [652 kB] 2023-09-30T00:49:59.474 INFO:teuthology.orchestra.run.smithi037.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-09-30T00:49:59.546 INFO:teuthology.orchestra.run.smithi050.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,464 kB] 2023-09-30T00:49:59.576 INFO:teuthology.orchestra.run.smithi037.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [652 kB] 2023-09-30T00:49:59.592 INFO:teuthology.orchestra.run.smithi050.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [885 kB] 2023-09-30T00:49:59.593 INFO:teuthology.orchestra.run.smithi050.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [385 kB] 2023-09-30T00:49:59.596 INFO:teuthology.orchestra.run.smithi050.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-09-30T00:49:59.596 INFO:teuthology.orchestra.run.smithi050.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,218 kB] 2023-09-30T00:49:59.616 INFO:teuthology.orchestra.run.smithi050.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-09-30T00:49:59.616 INFO:teuthology.orchestra.run.smithi050.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [309 kB] 2023-09-30T00:49:59.619 INFO:teuthology.orchestra.run.smithi050.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-09-30T00:49:59.619 INFO:teuthology.orchestra.run.smithi050.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [886 kB] 2023-09-30T00:49:59.626 INFO:teuthology.orchestra.run.smithi050.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [615 kB] 2023-09-30T00:49:59.633 INFO:teuthology.orchestra.run.smithi050.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [185 kB] 2023-09-30T00:49:59.634 INFO:teuthology.orchestra.run.smithi050.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-09-30T00:49:59.639 INFO:teuthology.orchestra.run.smithi050.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-09-30T00:49:59.639 INFO:teuthology.orchestra.run.smithi050.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-09-30T00:49:59.640 INFO:teuthology.orchestra.run.smithi050.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-09-30T00:49:59.640 INFO:teuthology.orchestra.run.smithi050.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-09-30T00:49:59.660 INFO:teuthology.orchestra.run.smithi050.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,852 kB] 2023-09-30T00:49:59.663 INFO:teuthology.orchestra.run.smithi037.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,464 kB] 2023-09-30T00:49:59.698 INFO:teuthology.orchestra.run.smithi050.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [469 kB] 2023-09-30T00:49:59.701 INFO:teuthology.orchestra.run.smithi050.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-09-30T00:49:59.702 INFO:teuthology.orchestra.run.smithi050.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,330 kB] 2023-09-30T00:49:59.707 INFO:teuthology.orchestra.run.smithi037.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [385 kB] 2023-09-30T00:49:59.707 INFO:teuthology.orchestra.run.smithi037.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-09-30T00:49:59.707 INFO:teuthology.orchestra.run.smithi037.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-09-30T00:49:59.707 INFO:teuthology.orchestra.run.smithi037.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,218 kB] 2023-09-30T00:49:59.718 INFO:teuthology.orchestra.run.smithi050.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.7 kB] 2023-09-30T00:49:59.718 INFO:teuthology.orchestra.run.smithi050.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [325 kB] 2023-09-30T00:49:59.721 INFO:teuthology.orchestra.run.smithi050.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-09-30T00:49:59.721 INFO:teuthology.orchestra.run.smithi050.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [748 kB] 2023-09-30T00:49:59.723 INFO:teuthology.orchestra.run.smithi037.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [309 kB] 2023-09-30T00:49:59.725 INFO:teuthology.orchestra.run.smithi037.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-09-30T00:49:59.725 INFO:teuthology.orchestra.run.smithi037.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [615 kB] 2023-09-30T00:49:59.726 INFO:teuthology.orchestra.run.smithi050.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,117 kB] 2023-09-30T00:49:59.729 INFO:teuthology.orchestra.run.smithi037.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [886 kB] 2023-09-30T00:49:59.734 INFO:teuthology.orchestra.run.smithi050.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [267 kB] 2023-09-30T00:49:59.735 INFO:teuthology.orchestra.run.smithi037.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [185 kB] 2023-09-30T00:49:59.736 INFO:teuthology.orchestra.run.smithi050.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-09-30T00:49:59.736 INFO:teuthology.orchestra.run.smithi050.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-09-30T00:49:59.736 INFO:teuthology.orchestra.run.smithi050.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.9 kB] 2023-09-30T00:49:59.736 INFO:teuthology.orchestra.run.smithi037.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-09-30T00:49:59.737 INFO:teuthology.orchestra.run.smithi037.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-09-30T00:49:59.737 INFO:teuthology.orchestra.run.smithi037.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-09-30T00:49:59.741 INFO:teuthology.orchestra.run.smithi037.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-09-30T00:49:59.742 INFO:teuthology.orchestra.run.smithi037.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-09-30T00:49:59.749 INFO:teuthology.orchestra.run.smithi050.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-09-30T00:49:59.749 INFO:teuthology.orchestra.run.smithi050.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-09-30T00:49:59.749 INFO:teuthology.orchestra.run.smithi050.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-09-30T00:49:59.750 INFO:teuthology.orchestra.run.smithi050.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-09-30T00:49:59.750 INFO:teuthology.orchestra.run.smithi050.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-09-30T00:49:59.765 INFO:teuthology.orchestra.run.smithi050.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-09-30T00:49:59.766 INFO:teuthology.orchestra.run.smithi050.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-09-30T00:49:59.766 INFO:teuthology.orchestra.run.smithi050.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-09-30T00:49:59.819 INFO:teuthology.orchestra.run.smithi037.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-09-30T00:49:59.960 INFO:teuthology.orchestra.run.smithi037.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [885 kB] 2023-09-30T00:50:00.206 INFO:teuthology.orchestra.run.smithi037.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,852 kB] 2023-09-30T00:50:00.370 INFO:teuthology.orchestra.run.smithi037.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [469 kB] 2023-09-30T00:50:00.383 INFO:teuthology.orchestra.run.smithi037.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-09-30T00:50:00.383 INFO:teuthology.orchestra.run.smithi037.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.7 kB] 2023-09-30T00:50:00.383 INFO:teuthology.orchestra.run.smithi037.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,330 kB] 2023-09-30T00:50:00.442 INFO:teuthology.orchestra.run.smithi037.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [325 kB] 2023-09-30T00:50:00.452 INFO:teuthology.orchestra.run.smithi037.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-09-30T00:50:00.616 INFO:teuthology.orchestra.run.smithi037.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,117 kB] 2023-09-30T00:50:01.248 INFO:teuthology.orchestra.run.smithi037.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [748 kB] 2023-09-30T00:50:01.286 INFO:teuthology.orchestra.run.smithi037.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [267 kB] 2023-09-30T00:50:01.300 INFO:teuthology.orchestra.run.smithi037.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-09-30T00:50:01.301 INFO:teuthology.orchestra.run.smithi037.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-09-30T00:50:01.301 INFO:teuthology.orchestra.run.smithi037.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.9 kB] 2023-09-30T00:50:01.302 INFO:teuthology.orchestra.run.smithi037.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-09-30T00:50:01.303 INFO:teuthology.orchestra.run.smithi037.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-09-30T00:50:01.303 INFO:teuthology.orchestra.run.smithi037.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-09-30T00:50:01.305 INFO:teuthology.orchestra.run.smithi037.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-09-30T00:50:01.331 INFO:teuthology.orchestra.run.smithi037.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-09-30T00:50:01.495 INFO:teuthology.orchestra.run.smithi037.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-09-30T00:50:01.678 INFO:teuthology.orchestra.run.smithi037.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-09-30T00:50:01.700 INFO:teuthology.orchestra.run.smithi037.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-09-30T00:50:03.505 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 17.4 MB in 3s (6,736 kB/s) 2023-09-30T00:50:03.674 INFO:teuthology.orchestra.run.smithi037.stdout:Fetched 17.4 MB in 2s (6,966 kB/s) 2023-09-30T00:50:04.905 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-09-30T00:50:04.931 DEBUG:teuthology.orchestra.run.smithi050:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-30T00:50:05.008 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-09-30T00:50:05.208 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2023-09-30T00:50:05.215 INFO:teuthology.orchestra.run.smithi050.stdout:Building dependency tree... 2023-09-30T00:50:05.215 INFO:teuthology.orchestra.run.smithi050.stdout:Reading state information... 2023-09-30T00:50:05.233 DEBUG:teuthology.orchestra.run.smithi037:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-30T00:50:05.301 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2023-09-30T00:50:05.420 INFO:teuthology.orchestra.run.smithi050.stdout:The following additional packages will be installed: 2023-09-30T00:50:05.420 INFO:teuthology.orchestra.run.smithi050.stdout: linux-generic linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-30T00:50:05.420 INFO:teuthology.orchestra.run.smithi050.stdout: linux-headers-generic linux-image-5.4.0-163-generic 2023-09-30T00:50:05.421 INFO:teuthology.orchestra.run.smithi050.stdout: linux-modules-5.4.0-163-generic linux-modules-extra-5.4.0-163-generic 2023-09-30T00:50:05.422 INFO:teuthology.orchestra.run.smithi050.stdout:Suggested packages: 2023-09-30T00:50:05.422 INFO:teuthology.orchestra.run.smithi050.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-09-30T00:50:05.422 INFO:teuthology.orchestra.run.smithi050.stdout:Recommended packages: 2023-09-30T00:50:05.422 INFO:teuthology.orchestra.run.smithi050.stdout: thermald 2023-09-30T00:50:05.485 INFO:teuthology.orchestra.run.smithi050.stdout:The following NEW packages will be installed: 2023-09-30T00:50:05.485 INFO:teuthology.orchestra.run.smithi050.stdout: linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-30T00:50:05.486 INFO:teuthology.orchestra.run.smithi050.stdout: linux-image-5.4.0-163-generic linux-modules-5.4.0-163-generic 2023-09-30T00:50:05.486 INFO:teuthology.orchestra.run.smithi050.stdout: linux-modules-extra-5.4.0-163-generic 2023-09-30T00:50:05.487 INFO:teuthology.orchestra.run.smithi050.stdout:The following packages will be upgraded: 2023-09-30T00:50:05.488 INFO:teuthology.orchestra.run.smithi050.stdout: linux-generic linux-headers-generic linux-image-generic 2023-09-30T00:50:05.507 INFO:teuthology.orchestra.run.smithi037.stdout:Building dependency tree... 2023-09-30T00:50:05.507 INFO:teuthology.orchestra.run.smithi037.stdout:Reading state information... 2023-09-30T00:50:05.543 INFO:teuthology.orchestra.run.smithi050.stdout:3 upgraded, 5 newly installed, 0 to remove and 263 not upgraded. 2023-09-30T00:50:05.543 INFO:teuthology.orchestra.run.smithi050.stdout:Need to get 77.1 MB of archives. 2023-09-30T00:50:05.544 INFO:teuthology.orchestra.run.smithi050.stdout:After this operation, 380 MB of additional disk space will be used. 2023-09-30T00:50:05.544 INFO:teuthology.orchestra.run.smithi050.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-163-generic amd64 5.4.0-163.180 [15.0 MB] 2023-09-30T00:50:05.700 INFO:teuthology.orchestra.run.smithi037.stdout:The following additional packages will be installed: 2023-09-30T00:50:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: linux-generic linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-30T00:50:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: linux-headers-generic linux-image-5.4.0-163-generic 2023-09-30T00:50:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: linux-modules-5.4.0-163-generic linux-modules-extra-5.4.0-163-generic 2023-09-30T00:50:05.701 INFO:teuthology.orchestra.run.smithi037.stdout:Suggested packages: 2023-09-30T00:50:05.701 INFO:teuthology.orchestra.run.smithi037.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-09-30T00:50:05.701 INFO:teuthology.orchestra.run.smithi037.stdout:Recommended packages: 2023-09-30T00:50:05.701 INFO:teuthology.orchestra.run.smithi037.stdout: thermald 2023-09-30T00:50:05.753 INFO:teuthology.orchestra.run.smithi037.stdout:The following NEW packages will be installed: 2023-09-30T00:50:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-30T00:50:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: linux-image-5.4.0-163-generic linux-modules-5.4.0-163-generic 2023-09-30T00:50:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: linux-modules-extra-5.4.0-163-generic 2023-09-30T00:50:05.755 INFO:teuthology.orchestra.run.smithi037.stdout:The following packages will be upgraded: 2023-09-30T00:50:05.755 INFO:teuthology.orchestra.run.smithi037.stdout: linux-generic linux-headers-generic linux-image-generic 2023-09-30T00:50:05.809 INFO:teuthology.orchestra.run.smithi037.stdout:3 upgraded, 5 newly installed, 0 to remove and 263 not upgraded. 2023-09-30T00:50:05.809 INFO:teuthology.orchestra.run.smithi037.stdout:Need to get 77.1 MB of archives. 2023-09-30T00:50:05.809 INFO:teuthology.orchestra.run.smithi037.stdout:After this operation, 380 MB of additional disk space will be used. 2023-09-30T00:50:05.809 INFO:teuthology.orchestra.run.smithi037.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-163-generic amd64 5.4.0-163.180 [15.0 MB] 2023-09-30T00:50:05.849 INFO:teuthology.orchestra.run.smithi050.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-163-generic amd64 5.4.0-163.180 [10.5 MB] 2023-09-30T00:50:05.944 INFO:teuthology.orchestra.run.smithi050.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-163-generic amd64 5.4.0-163.180 [39.2 MB] 2023-09-30T00:50:06.136 INFO:teuthology.orchestra.run.smithi037.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-163-generic amd64 5.4.0-163.180 [10.5 MB] 2023-09-30T00:50:06.228 INFO:teuthology.orchestra.run.smithi037.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-163-generic amd64 5.4.0-163.180 [39.2 MB] 2023-09-30T00:50:06.414 INFO:teuthology.orchestra.run.smithi050.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.163.160 [1,904 B] 2023-09-30T00:50:06.414 INFO:teuthology.orchestra.run.smithi050.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.163.160 [2,604 B] 2023-09-30T00:50:06.414 INFO:teuthology.orchestra.run.smithi050.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163 all 5.4.0-163.180 [11.0 MB] 2023-09-30T00:50:06.509 INFO:teuthology.orchestra.run.smithi050.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163-generic amd64 5.4.0-163.180 [1,358 kB] 2023-09-30T00:50:06.519 INFO:teuthology.orchestra.run.smithi050.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.163.160 [2,468 B] 2023-09-30T00:50:06.681 INFO:teuthology.orchestra.run.smithi037.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.163.160 [1,904 B] 2023-09-30T00:50:06.681 INFO:teuthology.orchestra.run.smithi037.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.163.160 [2,604 B] 2023-09-30T00:50:06.681 INFO:teuthology.orchestra.run.smithi037.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163 all 5.4.0-163.180 [11.0 MB] 2023-09-30T00:50:06.774 INFO:teuthology.orchestra.run.smithi037.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163-generic amd64 5.4.0-163.180 [1,358 kB] 2023-09-30T00:50:06.785 INFO:teuthology.orchestra.run.smithi037.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.163.160 [2,468 B] 2023-09-30T00:50:07.206 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 77.1 MB in 1s (75.3 MB/s) 2023-09-30T00:50:07.386 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package linux-modules-5.4.0-163-generic. 2023-09-30T00:50:07.521 INFO:teuthology.orchestra.run.smithi037.stdout:Fetched 77.1 MB in 1s (75.4 MB/s) 2023-09-30T00:50:07.721 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package linux-modules-5.4.0-163-generic. 2023-09-30T00:50:09.088 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-09-30T00:50:09.093 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../0-linux-modules-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:50:09.167 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:09.515 INFO:teuthology.orchestra.run.smithi037.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-09-30T00:50:09.520 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../0-linux-modules-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:50:09.693 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:11.194 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package linux-image-5.4.0-163-generic. 2023-09-30T00:50:11.208 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../1-linux-image-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:50:11.317 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:11.760 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package linux-image-5.4.0-163-generic. 2023-09-30T00:50:11.775 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../1-linux-image-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:50:11.880 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:12.422 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-163-generic. 2023-09-30T00:50:12.436 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:50:12.480 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:12.971 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-163-generic. 2023-09-30T00:50:12.988 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:50:13.029 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:17.569 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../3-linux-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:50:17.713 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:50:17.795 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../3-linux-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:50:17.936 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:50:18.048 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:50:18.175 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-image-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:50:18.246 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:50:18.406 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-image-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:50:18.486 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package linux-headers-5.4.0-163. 2023-09-30T00:50:18.500 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../5-linux-headers-5.4.0-163_5.4.0-163.180_all.deb ... 2023-09-30T00:50:18.544 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-30T00:50:18.734 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package linux-headers-5.4.0-163. 2023-09-30T00:50:18.748 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../5-linux-headers-5.4.0-163_5.4.0-163.180_all.deb ... 2023-09-30T00:50:18.792 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-30T00:50:23.911 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package linux-headers-5.4.0-163-generic. 2023-09-30T00:50:23.932 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../6-linux-headers-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:50:23.969 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:24.344 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package linux-headers-5.4.0-163-generic. 2023-09-30T00:50:24.366 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../6-linux-headers-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:50:24.410 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:27.062 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:50:27.208 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking linux-headers-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:50:27.234 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:50:27.361 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-headers-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:50:27.553 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:27.708 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:28.808 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-30T00:50:28.934 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:28.966 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-30T00:50:29.109 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:30.129 INFO:teuthology.orchestra.run.smithi050.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:50:30.129 INFO:teuthology.orchestra.run.smithi050.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-163-generic 2023-09-30T00:50:30.269 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:30.324 INFO:teuthology.orchestra.run.smithi037.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:50:30.325 INFO:teuthology.orchestra.run.smithi037.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-163-generic 2023-09-30T00:50:30.396 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:30.453 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:30.587 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:31.839 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-image-generic (5.4.0.163.160) ... 2023-09-30T00:50:31.950 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-headers-generic (5.4.0.163.160) ... 2023-09-30T00:50:32.049 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-generic (5.4.0.163.160) ... 2023-09-30T00:50:32.159 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:32.237 INFO:teuthology.orchestra.run.smithi037.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-09-30T00:50:32.238 INFO:teuthology.orchestra.run.smithi037.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:50:32.331 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-image-generic (5.4.0.163.160) ... 2023-09-30T00:50:32.474 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-headers-generic (5.4.0.163.160) ... 2023-09-30T00:50:32.600 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up linux-generic (5.4.0.163.160) ... 2023-09-30T00:50:32.726 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:50:32.794 INFO:teuthology.orchestra.run.smithi050.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-09-30T00:50:32.794 INFO:teuthology.orchestra.run.smithi050.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:50:55.299 INFO:teuthology.orchestra.run.smithi050.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-09-30T00:50:55.299 INFO:teuthology.orchestra.run.smithi050.stdout:Sourcing file `/etc/default/grub' 2023-09-30T00:50:55.312 INFO:teuthology.orchestra.run.smithi037.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-09-30T00:50:55.312 INFO:teuthology.orchestra.run.smithi037.stdout:Sourcing file `/etc/default/grub' 2023-09-30T00:50:55.316 INFO:teuthology.orchestra.run.smithi050.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-30T00:50:55.332 INFO:teuthology.orchestra.run.smithi037.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-30T00:50:55.347 INFO:teuthology.orchestra.run.smithi050.stdout:Generating grub configuration file ... 2023-09-30T00:50:55.349 INFO:teuthology.orchestra.run.smithi037.stdout:Generating grub configuration file ... 2023-09-30T00:50:55.694 INFO:teuthology.orchestra.run.smithi037.stdout:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:50:55.713 INFO:teuthology.orchestra.run.smithi037.stdout:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:50:55.722 INFO:teuthology.orchestra.run.smithi050.stdout:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:50:55.742 INFO:teuthology.orchestra.run.smithi050.stdout:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:50:56.083 INFO:teuthology.orchestra.run.smithi037.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-30T00:50:56.095 INFO:teuthology.orchestra.run.smithi037.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-30T00:50:56.165 INFO:teuthology.orchestra.run.smithi050.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-30T00:50:56.177 INFO:teuthology.orchestra.run.smithi050.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-30T00:50:58.568 INFO:teuthology.orchestra.run.smithi050.stdout:done 2023-09-30T00:50:58.700 INFO:teuthology.orchestra.run.smithi037.stdout:done 2023-09-30T00:50:59.001 DEBUG:teuthology.orchestra.run.smithi050:> dpkg -s linux-image-generic 2023-09-30T00:50:59.049 INFO:teuthology.orchestra.run.smithi050.stdout:Package: linux-image-generic 2023-09-30T00:50:59.049 INFO:teuthology.orchestra.run.smithi050.stdout:Status: install ok installed 2023-09-30T00:50:59.049 INFO:teuthology.orchestra.run.smithi050.stdout:Priority: optional 2023-09-30T00:50:59.050 INFO:teuthology.orchestra.run.smithi050.stdout:Section: kernel 2023-09-30T00:50:59.050 INFO:teuthology.orchestra.run.smithi050.stdout:Installed-Size: 20 2023-09-30T00:50:59.050 INFO:teuthology.orchestra.run.smithi050.stdout:Maintainer: Ubuntu Kernel Team 2023-09-30T00:50:59.050 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: amd64 2023-09-30T00:50:59.050 INFO:teuthology.orchestra.run.smithi050.stdout:Source: linux-meta 2023-09-30T00:50:59.050 INFO:teuthology.orchestra.run.smithi050.stdout:Version: 5.4.0.163.160 2023-09-30T00:50:59.050 INFO:teuthology.orchestra.run.smithi050.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-30T00:50:59.050 INFO:teuthology.orchestra.run.smithi050.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-30T00:50:59.050 INFO:teuthology.orchestra.run.smithi050.stdout:Recommends: thermald 2023-09-30T00:50:59.050 INFO:teuthology.orchestra.run.smithi050.stdout:Description: Generic Linux kernel image 2023-09-30T00:50:59.051 INFO:teuthology.orchestra.run.smithi050.stdout: This package will always depend on the latest generic kernel image 2023-09-30T00:50:59.051 INFO:teuthology.orchestra.run.smithi050.stdout: available. 2023-09-30T00:50:59.051 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-163-generic 2023-09-30T00:50:59.051 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-30T00:50:59.051 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-30T00:50:59.051 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-09-30T00:50:59.051 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=None, version=distro) 2023-09-30T00:50:59.051 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-get clean 2023-09-30T00:50:59.130 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-get update 2023-09-30T00:50:59.151 DEBUG:teuthology.orchestra.run.smithi037:> dpkg -s linux-image-generic 2023-09-30T00:50:59.195 INFO:teuthology.orchestra.run.smithi037.stdout:Package: linux-image-generic 2023-09-30T00:50:59.195 INFO:teuthology.orchestra.run.smithi037.stdout:Status: install ok installed 2023-09-30T00:50:59.195 INFO:teuthology.orchestra.run.smithi037.stdout:Priority: optional 2023-09-30T00:50:59.195 INFO:teuthology.orchestra.run.smithi037.stdout:Section: kernel 2023-09-30T00:50:59.196 INFO:teuthology.orchestra.run.smithi037.stdout:Installed-Size: 20 2023-09-30T00:50:59.196 INFO:teuthology.orchestra.run.smithi037.stdout:Maintainer: Ubuntu Kernel Team 2023-09-30T00:50:59.196 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: amd64 2023-09-30T00:50:59.196 INFO:teuthology.orchestra.run.smithi037.stdout:Source: linux-meta 2023-09-30T00:50:59.196 INFO:teuthology.orchestra.run.smithi037.stdout:Version: 5.4.0.163.160 2023-09-30T00:50:59.196 INFO:teuthology.orchestra.run.smithi037.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-30T00:50:59.196 INFO:teuthology.orchestra.run.smithi037.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-30T00:50:59.196 INFO:teuthology.orchestra.run.smithi037.stdout:Recommends: thermald 2023-09-30T00:50:59.196 INFO:teuthology.orchestra.run.smithi037.stdout:Description: Generic Linux kernel image 2023-09-30T00:50:59.196 INFO:teuthology.orchestra.run.smithi037.stdout: This package will always depend on the latest generic kernel image 2023-09-30T00:50:59.196 INFO:teuthology.orchestra.run.smithi037.stdout: available. 2023-09-30T00:50:59.197 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-163-generic 2023-09-30T00:50:59.197 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-30T00:50:59.197 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-30T00:50:59.197 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-09-30T00:50:59.197 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi037.front.sepia.ceph.com, path=None, version=distro) 2023-09-30T00:50:59.197 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-get clean 2023-09-30T00:50:59.279 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-get update 2023-09-30T00:50:59.354 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-30T00:50:59.380 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:50:59.417 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-30T00:50:59.463 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-30T00:50:59.528 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:50:59.545 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-30T00:50:59.610 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-30T00:50:59.692 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-30T00:51:01.125 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-09-30T00:51:01.150 DEBUG:teuthology.orchestra.run.smithi050:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-30T00:51:01.192 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2023-09-30T00:51:01.217 DEBUG:teuthology.orchestra.run.smithi037:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-30T00:51:01.219 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-09-30T00:51:01.285 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2023-09-30T00:51:01.424 INFO:teuthology.orchestra.run.smithi050.stdout:Building dependency tree... 2023-09-30T00:51:01.424 INFO:teuthology.orchestra.run.smithi050.stdout:Reading state information... 2023-09-30T00:51:01.487 INFO:teuthology.orchestra.run.smithi037.stdout:Building dependency tree... 2023-09-30T00:51:01.487 INFO:teuthology.orchestra.run.smithi037.stdout:Reading state information... 2023-09-30T00:51:01.651 INFO:teuthology.orchestra.run.smithi050.stdout:linux-image-generic is already the newest version (5.4.0.163.160). 2023-09-30T00:51:01.651 INFO:teuthology.orchestra.run.smithi050.stdout:0 upgraded, 0 newly installed, 0 to remove and 263 not upgraded. 2023-09-30T00:51:01.653 DEBUG:teuthology.orchestra.run.smithi050:> dpkg -s linux-image-generic 2023-09-30T00:51:01.673 INFO:teuthology.orchestra.run.smithi050.stdout:Package: linux-image-generic 2023-09-30T00:51:01.674 INFO:teuthology.orchestra.run.smithi050.stdout:Status: install ok installed 2023-09-30T00:51:01.674 INFO:teuthology.orchestra.run.smithi050.stdout:Priority: optional 2023-09-30T00:51:01.674 INFO:teuthology.orchestra.run.smithi050.stdout:Section: kernel 2023-09-30T00:51:01.674 INFO:teuthology.orchestra.run.smithi050.stdout:Installed-Size: 20 2023-09-30T00:51:01.674 INFO:teuthology.orchestra.run.smithi050.stdout:Maintainer: Ubuntu Kernel Team 2023-09-30T00:51:01.674 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: amd64 2023-09-30T00:51:01.674 INFO:teuthology.orchestra.run.smithi050.stdout:Source: linux-meta 2023-09-30T00:51:01.674 INFO:teuthology.orchestra.run.smithi050.stdout:Version: 5.4.0.163.160 2023-09-30T00:51:01.674 INFO:teuthology.orchestra.run.smithi050.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-30T00:51:01.675 INFO:teuthology.orchestra.run.smithi050.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-30T00:51:01.675 INFO:teuthology.orchestra.run.smithi050.stdout:Recommends: thermald 2023-09-30T00:51:01.675 INFO:teuthology.orchestra.run.smithi050.stdout:Description: Generic Linux kernel image 2023-09-30T00:51:01.675 INFO:teuthology.orchestra.run.smithi050.stdout: This package will always depend on the latest generic kernel image 2023-09-30T00:51:01.675 INFO:teuthology.orchestra.run.smithi050.stdout: available. 2023-09-30T00:51:01.675 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2023-09-30T00:51:01.681 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.4U0o5wMpKN 2023-09-30T00:51:01.682 DEBUG:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub/grub.cfg /tmp/tmp.4U0o5wMpKN 2023-09-30T00:51:01.693 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.4U0o5wMpKN 2023-09-30T00:51:01.711 INFO:teuthology.orchestra.run.smithi037.stdout:linux-image-generic is already the newest version (5.4.0.163.160). 2023-09-30T00:51:01.712 INFO:teuthology.orchestra.run.smithi037.stdout:0 upgraded, 0 newly installed, 0 to remove and 263 not upgraded. 2023-09-30T00:51:01.715 DEBUG:teuthology.orchestra.run.smithi037:> dpkg -s linux-image-generic 2023-09-30T00:51:01.735 INFO:teuthology.orchestra.run.smithi037.stdout:Package: linux-image-generic 2023-09-30T00:51:01.736 INFO:teuthology.orchestra.run.smithi037.stdout:Status: install ok installed 2023-09-30T00:51:01.736 INFO:teuthology.orchestra.run.smithi037.stdout:Priority: optional 2023-09-30T00:51:01.736 INFO:teuthology.orchestra.run.smithi037.stdout:Section: kernel 2023-09-30T00:51:01.736 INFO:teuthology.orchestra.run.smithi037.stdout:Installed-Size: 20 2023-09-30T00:51:01.736 INFO:teuthology.orchestra.run.smithi037.stdout:Maintainer: Ubuntu Kernel Team 2023-09-30T00:51:01.736 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: amd64 2023-09-30T00:51:01.736 INFO:teuthology.orchestra.run.smithi037.stdout:Source: linux-meta 2023-09-30T00:51:01.736 INFO:teuthology.orchestra.run.smithi037.stdout:Version: 5.4.0.163.160 2023-09-30T00:51:01.736 INFO:teuthology.orchestra.run.smithi037.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-30T00:51:01.736 INFO:teuthology.orchestra.run.smithi037.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-30T00:51:01.737 INFO:teuthology.orchestra.run.smithi037.stdout:Recommends: thermald 2023-09-30T00:51:01.737 INFO:teuthology.orchestra.run.smithi037.stdout:Description: Generic Linux kernel image 2023-09-30T00:51:01.737 INFO:teuthology.orchestra.run.smithi037.stdout: This package will always depend on the latest generic kernel image 2023-09-30T00:51:01.737 INFO:teuthology.orchestra.run.smithi037.stdout: available. 2023-09-30T00:51:01.737 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2023-09-30T00:51:01.741 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.sZTlK8bSB2 2023-09-30T00:51:01.741 DEBUG:teuthology.orchestra.run.smithi037:> sudo cp /boot/grub/grub.cfg /tmp/tmp.sZTlK8bSB2 2023-09-30T00:51:01.752 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.sZTlK8bSB2 2023-09-30T00:51:01.770 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.4U0o5wMpKN is 9KB 2023-09-30T00:51:01.784 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.4U0o5wMpKN 2023-09-30T00:51:01.789 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-09-30T00:51:01.831 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.sZTlK8bSB2 is 9KB 2023-09-30T00:51:01.845 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T00:51:01.846 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-09-30T00:51:01.846 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-09-30T00:51:01.881 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-163-generic 2023-09-30T00:51:01.882 DEBUG:teuthology.orchestra.run.smithi050:> sudo update-grub 2023-09-30T00:51:01.885 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.sZTlK8bSB2 2023-09-30T00:51:01.891 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-09-30T00:51:01.948 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T00:51:01.949 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-09-30T00:51:01.949 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-09-30T00:51:01.980 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-163-generic 2023-09-30T00:51:01.980 DEBUG:teuthology.orchestra.run.smithi037:> sudo update-grub 2023-09-30T00:51:03.083 INFO:teuthology.orchestra.run.smithi050.stderr:Sourcing file `/etc/default/grub' 2023-09-30T00:51:03.090 INFO:teuthology.orchestra.run.smithi050.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-30T00:51:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2023-09-30T00:51:03.262 INFO:teuthology.orchestra.run.smithi037.stderr:Sourcing file `/etc/default/grub' 2023-09-30T00:51:03.272 INFO:teuthology.orchestra.run.smithi037.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-30T00:51:03.282 INFO:teuthology.orchestra.run.smithi037.stderr:Generating grub configuration file ... 2023-09-30T00:51:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:51:03.385 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:51:03.562 INFO:teuthology.orchestra.run.smithi037.stderr:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:51:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:51:03.789 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-30T00:51:03.801 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-30T00:51:03.924 INFO:teuthology.orchestra.run.smithi037.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-30T00:51:03.935 INFO:teuthology.orchestra.run.smithi037.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-30T00:51:05.179 INFO:teuthology.orchestra.run.smithi050.stderr:done 2023-09-30T00:51:05.181 DEBUG:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2023-09-30T00:51:05.390 INFO:teuthology.orchestra.run.smithi037.stderr:done 2023-09-30T00:51:05.391 DEBUG:teuthology.orchestra.run.smithi037:> sudo shutdown -r now 2023-09-30T00:51:35.214 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-09-30T00:51:35.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-09-30T00:51:35.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:51:35.394 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-09-30T00:51:35.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-09-30T00:51:35.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:51:53.594 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-09-30T00:51:53.850 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2023-09-30T00:52:02.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-09-30T00:52:02.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:52:02.852 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-09-30T00:52:02.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:52:21.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2023-09-30T00:52:33.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-09-30T00:52:33.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:52:36.314 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2023-09-30T00:52:51.330 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-09-30T00:52:51.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:52:51.538 DEBUG:teuthology.orchestra.run.smithi037:> true 2023-09-30T00:52:52.227 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-09-30T00:52:52.227 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-163-generic"... 2023-09-30T00:52:52.228 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2023-09-30T00:52:52.234 INFO:teuthology.orchestra.run.smithi037.stdout:5.4.0-163-generic 2023-09-30T00:52:52.234 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-163-generic vs 5.4.0-163-generic 2023-09-30T00:52:52.234 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-30T00:52:52.235 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-09-30T00:52:53.236 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-09-30T00:52:53.236 DEBUG:teuthology.orchestra.run.smithi037:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-30T00:52:53.325 INFO:teuthology.orchestra.run.smithi037.stdout:ttyS1 2023-09-30T00:52:53.356 DEBUG:teuthology.parallel:result is None 2023-09-30T00:53:02.605 DEBUG:teuthology.orchestra.remote:timed out 2023-09-30T00:53:14.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-09-30T00:53:14.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:53:14.810 DEBUG:teuthology.orchestra.run.smithi050:> true 2023-09-30T00:53:15.514 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-09-30T00:53:15.514 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-163-generic"... 2023-09-30T00:53:15.514 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2023-09-30T00:53:15.521 INFO:teuthology.orchestra.run.smithi050.stdout:5.4.0-163-generic 2023-09-30T00:53:15.521 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-163-generic vs 5.4.0-163-generic 2023-09-30T00:53:15.521 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-30T00:53:15.521 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-09-30T00:53:16.522 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-09-30T00:53:16.522 DEBUG:teuthology.orchestra.run.smithi050:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-30T00:53:16.620 INFO:teuthology.orchestra.run.smithi050.stdout:ttyS1 2023-09-30T00:53:16.644 DEBUG:teuthology.parallel:result is None 2023-09-30T00:53:16.644 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-30T00:53:16.654 INFO:teuthology.task.internal:Creating test directory... 2023-09-30T00:53:16.654 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T00:53:16.656 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T00:53:16.664 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-30T00:53:16.675 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-30T00:53:16.689 INFO:teuthology.task.internal:Creating archive directory... 2023-09-30T00:53:16.689 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T00:53:16.691 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T00:53:16.709 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-30T00:53:16.720 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-30T00:53:16.721 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T00:53:16.747 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T00:53:16.765 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T00:53:16.772 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T00:53:16.777 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T00:53:16.783 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T00:53:16.785 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-30T00:53:16.796 INFO:teuthology.task.internal:Configuring sudo... 2023-09-30T00:53:16.797 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T00:53:16.799 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T00:53:16.816 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-30T00:53:16.829 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-30T00:53:16.829 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T00:53:16.831 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T00:53:16.864 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T00:53:16.884 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T00:53:16.891 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T00:53:16.891 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T00:53:16.921 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T00:53:16.928 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T00:53:16.935 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T00:53:16.935 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T00:53:16.961 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2023-09-30T00:53:16.963 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2023-09-30T00:53:17.054 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-30T00:53:17.063 INFO:teuthology.task.internal:Starting timer... 2023-09-30T00:53:17.063 INFO:teuthology.run_tasks:Running task pcp... 2023-09-30T00:53:17.080 INFO:teuthology.run_tasks:Running task selinux... 2023-09-30T00:53:17.094 DEBUG:teuthology.task.selinux:Excluding smithi037: OS 'ubuntu' does not support SELinux 2023-09-30T00:53:17.094 DEBUG:teuthology.task.selinux:Excluding smithi050: OS 'ubuntu' does not support SELinux 2023-09-30T00:53:17.094 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-30T00:53:17.094 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-09-30T00:53:17.094 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-30T00:53:17.095 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-30T00:53:17.110 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-30T00:53:17.116 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-30T00:53:17.242 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-30T00:53:17.262 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-30T00:53:17.263 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi050.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-30T00:59:08.890 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi050.front.sepia.ceph.com')] 2023-09-30T00:59:08.891 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-09-30T00:59:08.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:59:08.942 DEBUG:teuthology.orchestra.run.smithi037:> true 2023-09-30T00:59:09.068 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-09-30T00:59:09.068 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-09-30T00:59:09.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:59:09.120 DEBUG:teuthology.orchestra.run.smithi050:> true 2023-09-30T00:59:09.244 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-09-30T00:59:09.244 INFO:teuthology.run_tasks:Running task clock... 2023-09-30T00:59:09.257 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-30T00:59:09.257 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T00:59:09.257 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T00:59:09.261 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T00:59:09.261 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T00:59:09.295 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-09-30T00:59:09.295 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: Command line: ntpd -gq 2023-09-30T00:59:09.296 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: proto: precision = 0.120 usec (-23) 2023-09-30T00:59:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-30T00:59:09.297 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-30T00:59:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:restrict ::: KOD does nothing without LIMITED. 2023-09-30T00:59:09.298 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: restrict ::: KOD does nothing without LIMITED. 2023-09-30T00:59:09.299 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: Listen and drop on 0 v6wildcard [::]:123 2023-09-30T00:59:09.299 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-09-30T00:59:09.299 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: Listen normally on 2 lo 127.0.0.1:123 2023-09-30T00:59:09.299 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: Listen normally on 3 ens1f0 172.21.15.37:123 2023-09-30T00:59:09.299 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: Listen normally on 4 lo [::1]:123 2023-09-30T00:59:09.299 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:fe88:7794%3]:123 2023-09-30T00:59:09.299 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:09 ntpd[22677]: Listening on routing socket on fd #22 for interface updates 2023-09-30T00:59:09.320 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-09-30T00:59:09.321 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: Command line: ntpd -gq 2023-09-30T00:59:09.322 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: proto: precision = 0.095 usec (-23) 2023-09-30T00:59:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-30T00:59:09.322 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-30T00:59:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:restrict ::: KOD does nothing without LIMITED. 2023-09-30T00:59:09.324 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: restrict ::: KOD does nothing without LIMITED. 2023-09-30T00:59:09.324 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: Listen and drop on 0 v6wildcard [::]:123 2023-09-30T00:59:09.324 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-09-30T00:59:09.324 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: Listen normally on 2 lo 127.0.0.1:123 2023-09-30T00:59:09.324 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: Listen normally on 3 ens1f0 172.21.15.50:123 2023-09-30T00:59:09.324 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: Listen normally on 4 lo [::1]:123 2023-09-30T00:59:09.324 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1584%4]:123 2023-09-30T00:59:09.325 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:09 ntpd[22645]: Listening on routing socket on fd #22 for interface updates 2023-09-30T00:59:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:10 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:10 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:10 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:10 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:10 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:10 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:10 ntpd[22677]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:10 ntpd[22677]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:10.321 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:10 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:10.321 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:10 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:10.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:10 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:10.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:10 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:10.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:10 ntpd[22645]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:10.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:10 ntpd[22645]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:11.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:11 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:11.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:11 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:11.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:11 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:11.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:11 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:11.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:11 ntpd[22645]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:11.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:11 ntpd[22645]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:12 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:12 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:12 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:12 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:12 ntpd[22677]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:12 ntpd[22677]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:12.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:12.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:12.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:12.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:12.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:12.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:12.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:12.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:12.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:12.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:12.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:12.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:12 ntpd[22645]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:13 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:13 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:13 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:13 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:13 ntpd[22677]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:13 ntpd[22677]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:13.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:13 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:13.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:13 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:13.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:13 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:13.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:13 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:13.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:13 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:13.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:13 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:13.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:13 ntpd[22645]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:13.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:13 ntpd[22645]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:14.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:14.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:14.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:14.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:14.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:14.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:14.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:14.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:14.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:14.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:14.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:14.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:14 ntpd[22677]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:14 ntpd[22645]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:15 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:15 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:15 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:15 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:15 ntpd[22677]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:15 ntpd[22677]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:15.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:15 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:15.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:15 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:15.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:15 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:15.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:15 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:15.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:15 ntpd[22645]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:15.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:15 ntpd[22645]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:16 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:16 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:16 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:16 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:16 ntpd[22677]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:16 ntpd[22677]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:16 ntpd[22677]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:16 ntpd[22677]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:16.298 INFO:teuthology.orchestra.run.smithi037.stdout:30 Sep 00:59:16 ntpd[22677]: ntpd: time slew +0.000793 s 2023-09-30T00:59:16.298 INFO:teuthology.orchestra.run.smithi037.stdout:ntpd: time slew +0.000793s 2023-09-30T00:59:16.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:16 ntpd[22677]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-09-30T00:59:16.298 INFO:teuthology.orchestra.run.smithi037.stderr:30 Sep 00:59:16 ntpd[22677]: can't open /var/log/ntpstats/loopstats.20230930: Permission denied 2023-09-30T00:59:16.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:16.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:16.322 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:16.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:16.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:16.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:16.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:16.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:16.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:59:16.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:59:16.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:59:16.323 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:59:16.324 INFO:teuthology.orchestra.run.smithi050.stdout:30 Sep 00:59:16 ntpd[22645]: ntpd: time slew +0.000472 s 2023-09-30T00:59:16.324 INFO:teuthology.orchestra.run.smithi050.stdout:ntpd: time slew +0.000472s 2023-09-30T00:59:16.324 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-09-30T00:59:16.324 INFO:teuthology.orchestra.run.smithi050.stderr:30 Sep 00:59:16 ntpd[22645]: can't open /var/log/ntpstats/loopstats.20230930: Permission denied 2023-09-30T00:59:16.361 INFO:teuthology.orchestra.run.smithi037.stdout: remote refid st t when poll reach delay offset jitter 2023-09-30T00:59:16.361 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================== 2023-09-30T00:59:16.361 INFO:teuthology.orchestra.run.smithi037.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:59:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:59:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:59:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:59:16.384 INFO:teuthology.orchestra.run.smithi050.stdout: remote refid st t when poll reach delay offset jitter 2023-09-30T00:59:16.384 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================== 2023-09-30T00:59:16.385 INFO:teuthology.orchestra.run.smithi050.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:59:16.385 INFO:teuthology.orchestra.run.smithi050.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:59:16.385 INFO:teuthology.orchestra.run.smithi050.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:59:16.385 INFO:teuthology.orchestra.run.smithi050.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:59:16.385 INFO:teuthology.run_tasks:Running task install... 2023-09-30T00:59:16.397 DEBUG:teuthology.task.install:project ceph 2023-09-30T00:59:16.398 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}} 2023-09-30T00:59:16.398 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'} 2023-09-30T00:59:16.398 INFO:teuthology.task.install:Using flavor: default 2023-09-30T00:59:16.404 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-30T00:59:16.404 INFO:teuthology.task.install:extra packages: [] 2023-09-30T00:59:16.404 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-key list | grep Ceph 2023-09-30T00:59:16.405 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-key list | grep Ceph 2023-09-30T00:59:16.508 INFO:teuthology.orchestra.run.smithi037.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-09-30T00:59:16.513 INFO:teuthology.orchestra.run.smithi050.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-09-30T00:59:16.569 INFO:teuthology.orchestra.run.smithi037.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-09-30T00:59:16.569 INFO:teuthology.orchestra.run.smithi037.stdout:uid [ unknown] Ceph.com (release key) 2023-09-30T00:59:16.570 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-09-30T00:59:16.571 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-30T00:59:16.575 INFO:teuthology.orchestra.run.smithi050.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-09-30T00:59:16.575 INFO:teuthology.orchestra.run.smithi050.stdout:uid [ unknown] Ceph.com (release key) 2023-09-30T00:59:16.576 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-09-30T00:59:16.576 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-30T00:59:16.797 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default/ 2023-09-30T00:59:16.798 INFO:teuthology.task.install.deb:Package version is 17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:59:16.812 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default/ 2023-09-30T00:59:16.812 INFO:teuthology.task.install.deb:Package version is 17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:59:16.926 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T00:59:16.926 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-09-30T00:59:16.942 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-get update 2023-09-30T00:59:16.991 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T00:59:16.991 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-09-30T00:59:17.010 DEBUG:teuthology.orchestra.run.smithi050:> sudo apt-get update 2023-09-30T00:59:17.131 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-30T00:59:17.131 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:59:17.138 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-30T00:59:17.155 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-30T00:59:17.184 INFO:teuthology.orchestra.run.smithi037.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal InRelease 2023-09-30T00:59:17.205 INFO:teuthology.orchestra.run.smithi037.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-09-30T00:59:17.228 INFO:teuthology.orchestra.run.smithi037.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release.gpg 2023-09-30T00:59:17.249 INFO:teuthology.orchestra.run.smithi037.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main i386 Packages [3,375 B] 2023-09-30T00:59:17.300 INFO:teuthology.orchestra.run.smithi037.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-09-30T00:59:17.310 INFO:teuthology.orchestra.run.smithi050.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal InRelease 2023-09-30T00:59:17.324 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:59:17.324 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-30T00:59:17.346 INFO:teuthology.orchestra.run.smithi050.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-09-30T00:59:17.385 INFO:teuthology.orchestra.run.smithi050.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release.gpg 2023-09-30T00:59:17.394 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:6 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-30T00:59:17.422 INFO:teuthology.orchestra.run.smithi050.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-09-30T00:59:17.476 INFO:teuthology.orchestra.run.smithi050.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-30T00:59:17.487 INFO:teuthology.orchestra.run.smithi050.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main i386 Packages [3,375 B] 2023-09-30T00:59:17.807 INFO:teuthology.orchestra.run.smithi037.stdout:Fetched 28.4 kB in 1s (43.5 kB/s) 2023-09-30T00:59:18.006 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 28.4 kB in 1s (36.1 kB/s) 2023-09-30T00:59:18.869 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2023-09-30T00:59:18.895 DEBUG:teuthology.orchestra.run.smithi037:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1165-g93dce5e1-1focal cephadm=17.2.6-1165-g93dce5e1-1focal ceph-mds=17.2.6-1165-g93dce5e1-1focal ceph-mgr=17.2.6-1165-g93dce5e1-1focal ceph-common=17.2.6-1165-g93dce5e1-1focal ceph-fuse=17.2.6-1165-g93dce5e1-1focal ceph-test=17.2.6-1165-g93dce5e1-1focal ceph-volume=17.2.6-1165-g93dce5e1-1focal radosgw=17.2.6-1165-g93dce5e1-1focal python3-rados=17.2.6-1165-g93dce5e1-1focal python3-rgw=17.2.6-1165-g93dce5e1-1focal python3-cephfs=17.2.6-1165-g93dce5e1-1focal python3-rbd=17.2.6-1165-g93dce5e1-1focal libcephfs2=17.2.6-1165-g93dce5e1-1focal libcephfs-dev=17.2.6-1165-g93dce5e1-1focal librados2=17.2.6-1165-g93dce5e1-1focal librbd1=17.2.6-1165-g93dce5e1-1focal rbd-fuse=17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:59:18.969 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2023-09-30T00:59:19.073 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-09-30T00:59:19.097 DEBUG:teuthology.orchestra.run.smithi050:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1165-g93dce5e1-1focal cephadm=17.2.6-1165-g93dce5e1-1focal ceph-mds=17.2.6-1165-g93dce5e1-1focal ceph-mgr=17.2.6-1165-g93dce5e1-1focal ceph-common=17.2.6-1165-g93dce5e1-1focal ceph-fuse=17.2.6-1165-g93dce5e1-1focal ceph-test=17.2.6-1165-g93dce5e1-1focal ceph-volume=17.2.6-1165-g93dce5e1-1focal radosgw=17.2.6-1165-g93dce5e1-1focal python3-rados=17.2.6-1165-g93dce5e1-1focal python3-rgw=17.2.6-1165-g93dce5e1-1focal python3-cephfs=17.2.6-1165-g93dce5e1-1focal python3-rbd=17.2.6-1165-g93dce5e1-1focal libcephfs2=17.2.6-1165-g93dce5e1-1focal libcephfs-dev=17.2.6-1165-g93dce5e1-1focal librados2=17.2.6-1165-g93dce5e1-1focal librbd1=17.2.6-1165-g93dce5e1-1focal rbd-fuse=17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:59:19.169 INFO:teuthology.orchestra.run.smithi050.stdout:Reading package lists... 2023-09-30T00:59:19.176 INFO:teuthology.orchestra.run.smithi037.stdout:Building dependency tree... 2023-09-30T00:59:19.176 INFO:teuthology.orchestra.run.smithi037.stdout:Reading state information... 2023-09-30T00:59:19.354 INFO:teuthology.orchestra.run.smithi050.stdout:Building dependency tree... 2023-09-30T00:59:19.354 INFO:teuthology.orchestra.run.smithi050.stdout:Reading state information... 2023-09-30T00:59:19.365 INFO:teuthology.orchestra.run.smithi037.stdout:The following packages were automatically installed and are no longer required: 2023-09-30T00:59:19.366 INFO:teuthology.orchestra.run.smithi037.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-09-30T00:59:19.366 INFO:teuthology.orchestra.run.smithi037.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-30T00:59:19.366 INFO:teuthology.orchestra.run.smithi037.stdout:The following additional packages will be installed: 2023-09-30T00:59:19.366 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-09-30T00:59:19.366 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-09-30T00:59:19.366 INFO:teuthology.orchestra.run.smithi037.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-09-30T00:59:19.366 INFO:teuthology.orchestra.run.smithi037.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-09-30T00:59:19.366 INFO:teuthology.orchestra.run.smithi037.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-09-30T00:59:19.366 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-09-30T00:59:19.367 INFO:teuthology.orchestra.run.smithi037.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-09-30T00:59:19.367 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-09-30T00:59:19.367 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2023-09-30T00:59:19.367 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-09-30T00:59:19.367 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-09-30T00:59:19.367 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-09-30T00:59:19.367 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-09-30T00:59:19.368 INFO:teuthology.orchestra.run.smithi037.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-09-30T00:59:19.368 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-09-30T00:59:19.368 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2023-09-30T00:59:19.368 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2023-09-30T00:59:19.368 INFO:teuthology.orchestra.run.smithi037.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-09-30T00:59:19.368 INFO:teuthology.orchestra.run.smithi037.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2023-09-30T00:59:19.368 INFO:teuthology.orchestra.run.smithi037.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2023-09-30T00:59:19.368 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2023-09-30T00:59:19.368 INFO:teuthology.orchestra.run.smithi037.stdout: socat xmlstarlet 2023-09-30T00:59:19.369 INFO:teuthology.orchestra.run.smithi037.stdout:Suggested packages: 2023-09-30T00:59:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-09-30T00:59:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-09-30T00:59:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-09-30T00:59:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-09-30T00:59:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-09-30T00:59:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-09-30T00:59:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-09-30T00:59:19.370 INFO:teuthology.orchestra.run.smithi037.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-09-30T00:59:19.370 INFO:teuthology.orchestra.run.smithi037.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-09-30T00:59:19.370 INFO:teuthology.orchestra.run.smithi037.stdout:Recommended packages: 2023-09-30T00:59:19.370 INFO:teuthology.orchestra.run.smithi037.stdout: btrfs-tools 2023-09-30T00:59:19.490 INFO:teuthology.orchestra.run.smithi037.stdout:The following NEW packages will be installed: 2023-09-30T00:59:19.491 INFO:teuthology.orchestra.run.smithi037.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-09-30T00:59:19.491 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-09-30T00:59:19.491 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-09-30T00:59:19.491 INFO:teuthology.orchestra.run.smithi037.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2023-09-30T00:59:19.491 INFO:teuthology.orchestra.run.smithi037.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2023-09-30T00:59:19.491 INFO:teuthology.orchestra.run.smithi037.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2023-09-30T00:59:19.491 INFO:teuthology.orchestra.run.smithi037.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2023-09-30T00:59:19.493 INFO:teuthology.orchestra.run.smithi037.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2023-09-30T00:59:19.493 INFO:teuthology.orchestra.run.smithi037.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2023-09-30T00:59:19.493 INFO:teuthology.orchestra.run.smithi037.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-09-30T00:59:19.493 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2023-09-30T00:59:19.493 INFO:teuthology.orchestra.run.smithi037.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-09-30T00:59:19.494 INFO:teuthology.orchestra.run.smithi037.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-09-30T00:59:19.494 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2023-09-30T00:59:19.494 INFO:teuthology.orchestra.run.smithi037.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2023-09-30T00:59:19.494 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2023-09-30T00:59:19.494 INFO:teuthology.orchestra.run.smithi037.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-09-30T00:59:19.494 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2023-09-30T00:59:19.494 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2023-09-30T00:59:19.494 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-09-30T00:59:19.494 INFO:teuthology.orchestra.run.smithi037.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-09-30T00:59:19.494 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-09-30T00:59:19.494 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-09-30T00:59:19.495 INFO:teuthology.orchestra.run.smithi037.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-09-30T00:59:19.495 INFO:teuthology.orchestra.run.smithi037.stdout:The following packages will be upgraded: 2023-09-30T00:59:19.495 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 librbd1 2023-09-30T00:59:19.534 INFO:teuthology.orchestra.run.smithi050.stdout:The following packages were automatically installed and are no longer required: 2023-09-30T00:59:19.535 INFO:teuthology.orchestra.run.smithi050.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-09-30T00:59:19.535 INFO:teuthology.orchestra.run.smithi050.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-30T00:59:19.535 INFO:teuthology.orchestra.run.smithi050.stdout:The following additional packages will be installed: 2023-09-30T00:59:19.535 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-09-30T00:59:19.535 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-09-30T00:59:19.536 INFO:teuthology.orchestra.run.smithi050.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-09-30T00:59:19.536 INFO:teuthology.orchestra.run.smithi050.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-09-30T00:59:19.536 INFO:teuthology.orchestra.run.smithi050.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-09-30T00:59:19.537 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-09-30T00:59:19.537 INFO:teuthology.orchestra.run.smithi050.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-09-30T00:59:19.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-09-30T00:59:19.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2023-09-30T00:59:19.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-09-30T00:59:19.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-09-30T00:59:19.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-09-30T00:59:19.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-09-30T00:59:19.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-09-30T00:59:19.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-09-30T00:59:19.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2023-09-30T00:59:19.538 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2023-09-30T00:59:19.538 INFO:teuthology.orchestra.run.smithi050.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-09-30T00:59:19.538 INFO:teuthology.orchestra.run.smithi050.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2023-09-30T00:59:19.538 INFO:teuthology.orchestra.run.smithi050.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2023-09-30T00:59:19.538 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2023-09-30T00:59:19.538 INFO:teuthology.orchestra.run.smithi050.stdout: socat xmlstarlet 2023-09-30T00:59:19.539 INFO:teuthology.orchestra.run.smithi050.stdout:Suggested packages: 2023-09-30T00:59:19.539 INFO:teuthology.orchestra.run.smithi050.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-09-30T00:59:19.539 INFO:teuthology.orchestra.run.smithi050.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-09-30T00:59:19.539 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-09-30T00:59:19.539 INFO:teuthology.orchestra.run.smithi050.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-09-30T00:59:19.539 INFO:teuthology.orchestra.run.smithi050.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-09-30T00:59:19.539 INFO:teuthology.orchestra.run.smithi050.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-09-30T00:59:19.539 INFO:teuthology.orchestra.run.smithi050.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-09-30T00:59:19.540 INFO:teuthology.orchestra.run.smithi050.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-09-30T00:59:19.540 INFO:teuthology.orchestra.run.smithi050.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-09-30T00:59:19.540 INFO:teuthology.orchestra.run.smithi050.stdout:Recommended packages: 2023-09-30T00:59:19.540 INFO:teuthology.orchestra.run.smithi050.stdout: btrfs-tools 2023-09-30T00:59:19.556 INFO:teuthology.orchestra.run.smithi037.stdout:2 upgraded, 108 newly installed, 0 to remove and 245 not upgraded. 2023-09-30T00:59:19.556 INFO:teuthology.orchestra.run.smithi037.stdout:Need to get 209 MB of archives. 2023-09-30T00:59:19.556 INFO:teuthology.orchestra.run.smithi037.stdout:After this operation, 980 MB of additional disk space will be used. 2023-09-30T00:59:19.557 INFO:teuthology.orchestra.run.smithi037.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-09-30T00:59:19.639 INFO:teuthology.orchestra.run.smithi037.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-09-30T00:59:19.639 INFO:teuthology.orchestra.run.smithi037.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-09-30T00:59:19.649 INFO:teuthology.orchestra.run.smithi037.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-09-30T00:59:19.654 INFO:teuthology.orchestra.run.smithi037.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-09-30T00:59:19.660 INFO:teuthology.orchestra.run.smithi050.stdout:The following NEW packages will be installed: 2023-09-30T00:59:19.660 INFO:teuthology.orchestra.run.smithi050.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-09-30T00:59:19.660 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-09-30T00:59:19.661 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-09-30T00:59:19.661 INFO:teuthology.orchestra.run.smithi050.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2023-09-30T00:59:19.661 INFO:teuthology.orchestra.run.smithi050.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2023-09-30T00:59:19.661 INFO:teuthology.orchestra.run.smithi050.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2023-09-30T00:59:19.661 INFO:teuthology.orchestra.run.smithi050.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2023-09-30T00:59:19.661 INFO:teuthology.orchestra.run.smithi050.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2023-09-30T00:59:19.662 INFO:teuthology.orchestra.run.smithi050.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2023-09-30T00:59:19.662 INFO:teuthology.orchestra.run.smithi050.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-09-30T00:59:19.662 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2023-09-30T00:59:19.662 INFO:teuthology.orchestra.run.smithi050.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-09-30T00:59:19.662 INFO:teuthology.orchestra.run.smithi050.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-09-30T00:59:19.662 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2023-09-30T00:59:19.662 INFO:teuthology.orchestra.run.smithi050.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2023-09-30T00:59:19.662 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2023-09-30T00:59:19.662 INFO:teuthology.orchestra.run.smithi050.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-09-30T00:59:19.662 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2023-09-30T00:59:19.663 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2023-09-30T00:59:19.663 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-09-30T00:59:19.663 INFO:teuthology.orchestra.run.smithi050.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-09-30T00:59:19.663 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-09-30T00:59:19.663 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-09-30T00:59:19.663 INFO:teuthology.orchestra.run.smithi050.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-09-30T00:59:19.664 INFO:teuthology.orchestra.run.smithi050.stdout:The following packages will be upgraded: 2023-09-30T00:59:19.664 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 librbd1 2023-09-30T00:59:19.678 INFO:teuthology.orchestra.run.smithi037.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-09-30T00:59:19.679 INFO:teuthology.orchestra.run.smithi037.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-09-30T00:59:19.685 INFO:teuthology.orchestra.run.smithi037.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-09-30T00:59:19.716 INFO:teuthology.orchestra.run.smithi037.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-09-30T00:59:19.717 INFO:teuthology.orchestra.run.smithi037.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-09-30T00:59:19.718 INFO:teuthology.orchestra.run.smithi037.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1165-g93dce5e1-1focal [3,501 kB] 2023-09-30T00:59:19.724 INFO:teuthology.orchestra.run.smithi037.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-09-30T00:59:19.725 INFO:teuthology.orchestra.run.smithi037.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-09-30T00:59:19.726 INFO:teuthology.orchestra.run.smithi037.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-09-30T00:59:19.728 INFO:teuthology.orchestra.run.smithi037.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-09-30T00:59:19.728 INFO:teuthology.orchestra.run.smithi037.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-09-30T00:59:19.729 INFO:teuthology.orchestra.run.smithi037.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-09-30T00:59:19.730 INFO:teuthology.orchestra.run.smithi037.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-09-30T00:59:19.733 INFO:teuthology.orchestra.run.smithi037.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-09-30T00:59:19.748 INFO:teuthology.orchestra.run.smithi037.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-09-30T00:59:19.748 INFO:teuthology.orchestra.run.smithi037.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-09-30T00:59:19.748 INFO:teuthology.orchestra.run.smithi037.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-09-30T00:59:19.750 INFO:teuthology.orchestra.run.smithi037.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-09-30T00:59:19.750 INFO:teuthology.orchestra.run.smithi037.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-09-30T00:59:19.750 INFO:teuthology.orchestra.run.smithi037.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-09-30T00:59:19.752 INFO:teuthology.orchestra.run.smithi037.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-09-30T00:59:19.752 INFO:teuthology.orchestra.run.smithi037.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-09-30T00:59:19.754 INFO:teuthology.orchestra.run.smithi037.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-09-30T00:59:19.766 INFO:teuthology.orchestra.run.smithi037.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-09-30T00:59:19.767 INFO:teuthology.orchestra.run.smithi037.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-09-30T00:59:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-09-30T00:59:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-09-30T00:59:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-09-30T00:59:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-09-30T00:59:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-09-30T00:59:19.790 INFO:teuthology.orchestra.run.smithi037.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-09-30T00:59:19.790 INFO:teuthology.orchestra.run.smithi037.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-09-30T00:59:19.790 INFO:teuthology.orchestra.run.smithi037.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-09-30T00:59:19.799 INFO:teuthology.orchestra.run.smithi037.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-09-30T00:59:19.801 INFO:teuthology.orchestra.run.smithi037.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-09-30T00:59:19.812 INFO:teuthology.orchestra.run.smithi050.stdout:2 upgraded, 108 newly installed, 0 to remove and 245 not upgraded. 2023-09-30T00:59:19.813 INFO:teuthology.orchestra.run.smithi050.stdout:Need to get 209 MB of archives. 2023-09-30T00:59:19.813 INFO:teuthology.orchestra.run.smithi050.stdout:After this operation, 980 MB of additional disk space will be used. 2023-09-30T00:59:19.813 INFO:teuthology.orchestra.run.smithi050.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1165-g93dce5e1-1focal [3,501 kB] 2023-09-30T00:59:19.816 INFO:teuthology.orchestra.run.smithi037.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-09-30T00:59:19.817 INFO:teuthology.orchestra.run.smithi037.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-09-30T00:59:19.818 INFO:teuthology.orchestra.run.smithi037.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-09-30T00:59:19.854 INFO:teuthology.orchestra.run.smithi050.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-09-30T00:59:19.902 INFO:teuthology.orchestra.run.smithi037.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-09-30T00:59:19.919 INFO:teuthology.orchestra.run.smithi037.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-09-30T00:59:19.920 INFO:teuthology.orchestra.run.smithi037.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-09-30T00:59:19.932 INFO:teuthology.orchestra.run.smithi037.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-09-30T00:59:19.932 INFO:teuthology.orchestra.run.smithi037.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-09-30T00:59:19.995 INFO:teuthology.orchestra.run.smithi037.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-09-30T00:59:19.996 INFO:teuthology.orchestra.run.smithi037.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-09-30T00:59:19.997 INFO:teuthology.orchestra.run.smithi037.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-09-30T00:59:19.998 INFO:teuthology.orchestra.run.smithi037.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-09-30T00:59:19.998 INFO:teuthology.orchestra.run.smithi037.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-09-30T00:59:20.000 INFO:teuthology.orchestra.run.smithi037.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-09-30T00:59:20.000 INFO:teuthology.orchestra.run.smithi037.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-09-30T00:59:20.012 INFO:teuthology.orchestra.run.smithi037.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-09-30T00:59:20.015 INFO:teuthology.orchestra.run.smithi037.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-09-30T00:59:20.017 INFO:teuthology.orchestra.run.smithi037.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-09-30T00:59:20.029 INFO:teuthology.orchestra.run.smithi037.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-09-30T00:59:20.029 INFO:teuthology.orchestra.run.smithi037.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-09-30T00:59:20.033 INFO:teuthology.orchestra.run.smithi037.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-09-30T00:59:20.036 INFO:teuthology.orchestra.run.smithi037.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1165-g93dce5e1-1focal [3,552 kB] 2023-09-30T00:59:20.037 INFO:teuthology.orchestra.run.smithi037.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-09-30T00:59:20.057 INFO:teuthology.orchestra.run.smithi037.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-09-30T00:59:20.060 INFO:teuthology.orchestra.run.smithi037.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-09-30T00:59:20.100 INFO:teuthology.orchestra.run.smithi037.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-09-30T00:59:20.101 INFO:teuthology.orchestra.run.smithi037.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-09-30T00:59:20.101 INFO:teuthology.orchestra.run.smithi037.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-09-30T00:59:20.101 INFO:teuthology.orchestra.run.smithi037.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-09-30T00:59:20.102 INFO:teuthology.orchestra.run.smithi037.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-09-30T00:59:20.103 INFO:teuthology.orchestra.run.smithi037.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-09-30T00:59:20.115 INFO:teuthology.orchestra.run.smithi037.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-09-30T00:59:20.115 INFO:teuthology.orchestra.run.smithi037.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-09-30T00:59:20.115 INFO:teuthology.orchestra.run.smithi037.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-09-30T00:59:20.117 INFO:teuthology.orchestra.run.smithi037.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-09-30T00:59:20.134 INFO:teuthology.orchestra.run.smithi037.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-09-30T00:59:20.135 INFO:teuthology.orchestra.run.smithi037.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-09-30T00:59:20.135 INFO:teuthology.orchestra.run.smithi037.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-09-30T00:59:20.136 INFO:teuthology.orchestra.run.smithi037.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-09-30T00:59:20.138 INFO:teuthology.orchestra.run.smithi037.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-09-30T00:59:20.139 INFO:teuthology.orchestra.run.smithi037.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-09-30T00:59:20.164 INFO:teuthology.orchestra.run.smithi037.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1165-g93dce5e1-1focal [758 kB] 2023-09-30T00:59:20.252 INFO:teuthology.orchestra.run.smithi050.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-09-30T00:59:20.259 INFO:teuthology.orchestra.run.smithi050.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-09-30T00:59:20.279 INFO:teuthology.orchestra.run.smithi037.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1165-g93dce5e1-1focal [414 kB] 2023-09-30T00:59:20.286 INFO:teuthology.orchestra.run.smithi037.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1165-g93dce5e1-1focal [22.5 kB] 2023-09-30T00:59:20.286 INFO:teuthology.orchestra.run.smithi037.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1165-g93dce5e1-1focal [200 kB] 2023-09-30T00:59:20.288 INFO:teuthology.orchestra.run.smithi037.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1165-g93dce5e1-1focal [47.1 kB] 2023-09-30T00:59:20.288 INFO:teuthology.orchestra.run.smithi037.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1165-g93dce5e1-1focal [385 kB] 2023-09-30T00:59:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1165-g93dce5e1-1focal [5,569 kB] 2023-09-30T00:59:20.303 INFO:teuthology.orchestra.run.smithi050.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-09-30T00:59:20.316 INFO:teuthology.orchestra.run.smithi050.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-09-30T00:59:20.322 INFO:teuthology.orchestra.run.smithi050.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1165-g93dce5e1-1focal [3,552 kB] 2023-09-30T00:59:20.444 INFO:teuthology.orchestra.run.smithi050.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-09-30T00:59:20.448 INFO:teuthology.orchestra.run.smithi050.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-09-30T00:59:20.478 INFO:teuthology.orchestra.run.smithi050.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-09-30T00:59:20.486 INFO:teuthology.orchestra.run.smithi037.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1165-g93dce5e1-1focal [116 kB] 2023-09-30T00:59:20.490 INFO:teuthology.orchestra.run.smithi037.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1165-g93dce5e1-1focal [480 kB] 2023-09-30T00:59:20.514 INFO:teuthology.orchestra.run.smithi037.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1165-g93dce5e1-1focal [27.7 MB] 2023-09-30T00:59:20.623 INFO:teuthology.orchestra.run.smithi050.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-09-30T00:59:20.631 INFO:teuthology.orchestra.run.smithi050.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-09-30T00:59:20.654 INFO:teuthology.orchestra.run.smithi050.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-09-30T00:59:20.699 INFO:teuthology.orchestra.run.smithi050.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1165-g93dce5e1-1focal [758 kB] 2023-09-30T00:59:20.782 INFO:teuthology.orchestra.run.smithi050.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1165-g93dce5e1-1focal [414 kB] 2023-09-30T00:59:20.824 INFO:teuthology.orchestra.run.smithi050.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-09-30T00:59:20.830 INFO:teuthology.orchestra.run.smithi050.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1165-g93dce5e1-1focal [22.5 kB] 2023-09-30T00:59:20.831 INFO:teuthology.orchestra.run.smithi050.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1165-g93dce5e1-1focal [200 kB] 2023-09-30T00:59:20.852 INFO:teuthology.orchestra.run.smithi050.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1165-g93dce5e1-1focal [47.1 kB] 2023-09-30T00:59:20.857 INFO:teuthology.orchestra.run.smithi050.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1165-g93dce5e1-1focal [385 kB] 2023-09-30T00:59:20.903 INFO:teuthology.orchestra.run.smithi050.stdout:Get:21 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1165-g93dce5e1-1focal [5,569 kB] 2023-09-30T00:59:21.072 INFO:teuthology.orchestra.run.smithi050.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-09-30T00:59:21.266 INFO:teuthology.orchestra.run.smithi050.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-09-30T00:59:21.285 INFO:teuthology.orchestra.run.smithi050.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-09-30T00:59:21.299 INFO:teuthology.orchestra.run.smithi050.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-09-30T00:59:21.351 INFO:teuthology.orchestra.run.smithi050.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-09-30T00:59:21.359 INFO:teuthology.orchestra.run.smithi050.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-09-30T00:59:21.362 INFO:teuthology.orchestra.run.smithi050.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-09-30T00:59:21.366 INFO:teuthology.orchestra.run.smithi050.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-09-30T00:59:21.369 INFO:teuthology.orchestra.run.smithi050.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-09-30T00:59:21.381 INFO:teuthology.orchestra.run.smithi050.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-09-30T00:59:21.534 INFO:teuthology.orchestra.run.smithi050.stdout:Get:32 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1165-g93dce5e1-1focal [116 kB] 2023-09-30T00:59:21.546 INFO:teuthology.orchestra.run.smithi050.stdout:Get:33 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1165-g93dce5e1-1focal [480 kB] 2023-09-30T00:59:21.546 INFO:teuthology.orchestra.run.smithi050.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-09-30T00:59:21.603 INFO:teuthology.orchestra.run.smithi050.stdout:Get:35 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1165-g93dce5e1-1focal [27.7 MB] 2023-09-30T00:59:21.630 INFO:teuthology.orchestra.run.smithi050.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-09-30T00:59:21.814 INFO:teuthology.orchestra.run.smithi050.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-09-30T00:59:21.849 INFO:teuthology.orchestra.run.smithi050.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-09-30T00:59:21.908 INFO:teuthology.orchestra.run.smithi037.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1165-g93dce5e1-1focal [6,453 kB] 2023-09-30T00:59:21.914 INFO:teuthology.orchestra.run.smithi050.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-09-30T00:59:21.954 INFO:teuthology.orchestra.run.smithi050.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-09-30T00:59:21.960 INFO:teuthology.orchestra.run.smithi050.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-09-30T00:59:22.071 INFO:teuthology.orchestra.run.smithi050.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-09-30T00:59:22.071 INFO:teuthology.orchestra.run.smithi050.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-09-30T00:59:22.075 INFO:teuthology.orchestra.run.smithi050.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-09-30T00:59:22.079 INFO:teuthology.orchestra.run.smithi050.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-09-30T00:59:22.098 INFO:teuthology.orchestra.run.smithi037.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1165-g93dce5e1-1focal [221 kB] 2023-09-30T00:59:22.107 INFO:teuthology.orchestra.run.smithi037.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1165-g93dce5e1-1focal [153 kB] 2023-09-30T00:59:22.115 INFO:teuthology.orchestra.run.smithi037.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1165-g93dce5e1-1focal [1,589 kB] 2023-09-30T00:59:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1165-g93dce5e1-1focal [7,845 kB] 2023-09-30T00:59:22.255 INFO:teuthology.orchestra.run.smithi050.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-09-30T00:59:22.544 INFO:teuthology.orchestra.run.smithi037.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1165-g93dce5e1-1focal [28.7 MB] 2023-09-30T00:59:22.669 INFO:teuthology.orchestra.run.smithi050.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-09-30T00:59:22.701 INFO:teuthology.orchestra.run.smithi050.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-09-30T00:59:22.705 INFO:teuthology.orchestra.run.smithi050.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-09-30T00:59:22.754 INFO:teuthology.orchestra.run.smithi050.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-09-30T00:59:22.788 INFO:teuthology.orchestra.run.smithi050.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-09-30T00:59:22.813 INFO:teuthology.orchestra.run.smithi050.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-09-30T00:59:22.816 INFO:teuthology.orchestra.run.smithi050.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-09-30T00:59:22.824 INFO:teuthology.orchestra.run.smithi050.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-09-30T00:59:23.329 INFO:teuthology.orchestra.run.smithi050.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-09-30T00:59:23.350 INFO:teuthology.orchestra.run.smithi050.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-09-30T00:59:23.516 INFO:teuthology.orchestra.run.smithi050.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-09-30T00:59:23.976 INFO:teuthology.orchestra.run.smithi037.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1165-g93dce5e1-1focal [3,928 B] 2023-09-30T00:59:23.977 INFO:teuthology.orchestra.run.smithi037.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1165-g93dce5e1-1focal [868 kB] 2023-09-30T00:59:23.990 INFO:teuthology.orchestra.run.smithi037.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1165-g93dce5e1-1focal [2,404 kB] 2023-09-30T00:59:24.008 INFO:teuthology.orchestra.run.smithi037.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1165-g93dce5e1-1focal [78.9 kB] 2023-09-30T00:59:24.008 INFO:teuthology.orchestra.run.smithi037.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1165-g93dce5e1-1focal [117 kB] 2023-09-30T00:59:24.009 INFO:teuthology.orchestra.run.smithi037.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1165-g93dce5e1-1focal [1,760 kB] 2023-09-30T00:59:24.076 INFO:teuthology.orchestra.run.smithi037.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1165-g93dce5e1-1focal [7,861 kB] 2023-09-30T00:59:24.202 INFO:teuthology.orchestra.run.smithi050.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-09-30T00:59:24.202 INFO:teuthology.orchestra.run.smithi050.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-09-30T00:59:24.204 INFO:teuthology.orchestra.run.smithi050.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-09-30T00:59:24.206 INFO:teuthology.orchestra.run.smithi050.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-09-30T00:59:24.207 INFO:teuthology.orchestra.run.smithi050.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-09-30T00:59:24.208 INFO:teuthology.orchestra.run.smithi050.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-09-30T00:59:24.210 INFO:teuthology.orchestra.run.smithi050.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-09-30T00:59:24.212 INFO:teuthology.orchestra.run.smithi050.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-09-30T00:59:24.213 INFO:teuthology.orchestra.run.smithi050.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-09-30T00:59:24.283 INFO:teuthology.orchestra.run.smithi050.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-09-30T00:59:24.430 INFO:teuthology.orchestra.run.smithi037.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1165-g93dce5e1-1focal [4,048 B] 2023-09-30T00:59:24.430 INFO:teuthology.orchestra.run.smithi037.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1165-g93dce5e1-1focal [56.2 MB] 2023-09-30T00:59:24.456 INFO:teuthology.orchestra.run.smithi050.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-09-30T00:59:24.836 INFO:teuthology.orchestra.run.smithi050.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-09-30T00:59:24.887 INFO:teuthology.orchestra.run.smithi050.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-09-30T00:59:24.966 INFO:teuthology.orchestra.run.smithi050.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-09-30T00:59:25.040 INFO:teuthology.orchestra.run.smithi050.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-09-30T00:59:25.041 INFO:teuthology.orchestra.run.smithi050.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1165-g93dce5e1-1focal [6,453 kB] 2023-09-30T00:59:25.071 INFO:teuthology.orchestra.run.smithi050.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-09-30T00:59:25.225 INFO:teuthology.orchestra.run.smithi050.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-09-30T00:59:25.234 INFO:teuthology.orchestra.run.smithi050.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-09-30T00:59:25.383 INFO:teuthology.orchestra.run.smithi050.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-09-30T00:59:25.384 INFO:teuthology.orchestra.run.smithi050.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-09-30T00:59:25.384 INFO:teuthology.orchestra.run.smithi050.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-09-30T00:59:25.572 INFO:teuthology.orchestra.run.smithi050.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-09-30T00:59:25.655 INFO:teuthology.orchestra.run.smithi050.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-09-30T00:59:25.767 INFO:teuthology.orchestra.run.smithi050.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1165-g93dce5e1-1focal [221 kB] 2023-09-30T00:59:25.795 INFO:teuthology.orchestra.run.smithi050.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1165-g93dce5e1-1focal [153 kB] 2023-09-30T00:59:25.815 INFO:teuthology.orchestra.run.smithi050.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1165-g93dce5e1-1focal [1,589 kB] 2023-09-30T00:59:25.837 INFO:teuthology.orchestra.run.smithi050.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-09-30T00:59:26.026 INFO:teuthology.orchestra.run.smithi050.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1165-g93dce5e1-1focal [7,845 kB] 2023-09-30T00:59:26.172 INFO:teuthology.orchestra.run.smithi050.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-09-30T00:59:26.172 INFO:teuthology.orchestra.run.smithi050.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-09-30T00:59:26.177 INFO:teuthology.orchestra.run.smithi050.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-09-30T00:59:26.178 INFO:teuthology.orchestra.run.smithi050.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-09-30T00:59:26.188 INFO:teuthology.orchestra.run.smithi050.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-09-30T00:59:26.192 INFO:teuthology.orchestra.run.smithi050.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-09-30T00:59:26.194 INFO:teuthology.orchestra.run.smithi050.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-09-30T00:59:26.194 INFO:teuthology.orchestra.run.smithi050.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-09-30T00:59:26.372 INFO:teuthology.orchestra.run.smithi050.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-09-30T00:59:26.631 INFO:teuthology.orchestra.run.smithi050.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-09-30T00:59:27.038 INFO:teuthology.orchestra.run.smithi050.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1165-g93dce5e1-1focal [28.7 MB] 2023-09-30T00:59:27.187 INFO:teuthology.orchestra.run.smithi037.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1165-g93dce5e1-1focal [229 kB] 2023-09-30T00:59:27.192 INFO:teuthology.orchestra.run.smithi037.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1165-g93dce5e1-1focal [21.9 kB] 2023-09-30T00:59:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1165-g93dce5e1-1focal [17.0 MB] 2023-09-30T00:59:27.917 INFO:teuthology.orchestra.run.smithi037.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1165-g93dce5e1-1focal [79.5 kB] 2023-09-30T00:59:28.568 INFO:teuthology.orchestra.run.smithi037.stdout:Fetched 209 MB in 8s (24.9 MB/s) 2023-09-30T00:59:28.619 INFO:teuthology.orchestra.run.smithi050.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1165-g93dce5e1-1focal [3,928 B] 2023-09-30T00:59:28.693 INFO:teuthology.orchestra.run.smithi050.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1165-g93dce5e1-1focal [868 kB] 2023-09-30T00:59:28.709 INFO:teuthology.orchestra.run.smithi050.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1165-g93dce5e1-1focal [2,404 kB] 2023-09-30T00:59:28.742 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-09-30T00:59:28.752 INFO:teuthology.orchestra.run.smithi050.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1165-g93dce5e1-1focal [78.9 kB] 2023-09-30T00:59:28.765 INFO:teuthology.orchestra.run.smithi050.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1165-g93dce5e1-1focal [117 kB] 2023-09-30T00:59:28.766 INFO:teuthology.orchestra.run.smithi050.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1165-g93dce5e1-1focal [1,760 kB] 2023-09-30T00:59:28.798 INFO:teuthology.orchestra.run.smithi037.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141758 files and directories currently installed.) 2023-09-30T00:59:28.802 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-09-30T00:59:28.831 INFO:teuthology.orchestra.run.smithi050.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1165-g93dce5e1-1focal [7,861 kB] 2023-09-30T00:59:28.924 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-30T00:59:29.127 INFO:teuthology.orchestra.run.smithi050.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1165-g93dce5e1-1focal [4,048 B] 2023-09-30T00:59:29.127 INFO:teuthology.orchestra.run.smithi050.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1165-g93dce5e1-1focal [56.2 MB] 2023-09-30T00:59:29.303 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../001-librbd1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:29.453 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking librbd1 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-30T00:59:30.098 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../002-librados2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:30.198 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking librados2 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-30T00:59:30.961 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libcephfs2. 2023-09-30T00:59:30.977 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../003-libcephfs2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:31.019 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:31.388 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-rados. 2023-09-30T00:59:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../004-python3-rados_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:31.446 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:31.465 INFO:teuthology.orchestra.run.smithi050.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1165-g93dce5e1-1focal [229 kB] 2023-09-30T00:59:31.469 INFO:teuthology.orchestra.run.smithi050.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1165-g93dce5e1-1focal [21.9 kB] 2023-09-30T00:59:31.533 INFO:teuthology.orchestra.run.smithi050.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1165-g93dce5e1-1focal [17.0 MB] 2023-09-30T00:59:31.783 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-09-30T00:59:31.802 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:59:31.832 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:32.094 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-cephfs. 2023-09-30T00:59:32.110 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../006-python3-cephfs_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:32.138 INFO:teuthology.orchestra.run.smithi050.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1165-g93dce5e1-1focal [79.5 kB] 2023-09-30T00:59:32.152 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:32.521 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-ceph-common. 2023-09-30T00:59:32.541 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:59:32.579 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:32.787 INFO:teuthology.orchestra.run.smithi050.stdout:Fetched 209 MB in 12s (16.8 MB/s) 2023-09-30T00:59:32.924 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-prettytable. 2023-09-30T00:59:32.928 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-09-30T00:59:32.940 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2023-09-30T00:59:32.982 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-09-30T00:59:32.985 INFO:teuthology.orchestra.run.smithi050.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141758 files and directories currently installed.) 2023-09-30T00:59:32.989 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-09-30T00:59:33.060 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-30T00:59:33.227 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-rbd. 2023-09-30T00:59:33.244 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../009-python3-rbd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:33.276 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:33.407 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../001-librbd1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:33.532 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librbd1 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-30T00:59:33.629 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-09-30T00:59:33.645 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-09-30T00:59:33.687 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-30T00:59:34.074 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-09-30T00:59:34.091 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2023-09-30T00:59:34.157 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-09-30T00:59:34.195 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../002-librados2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:34.296 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librados2 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-30T00:59:34.504 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-09-30T00:59:34.520 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-09-30T00:59:34.562 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-30T00:59:35.000 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-09-30T00:59:35.009 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libcephfs2. 2023-09-30T00:59:35.015 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-09-30T00:59:35.027 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../003-libcephfs2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:35.057 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-30T00:59:35.067 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:35.387 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rados. 2023-09-30T00:59:35.404 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../004-python3-rados_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:35.436 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:35.441 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-09-30T00:59:35.462 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-09-30T00:59:35.499 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-30T00:59:35.773 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-09-30T00:59:35.793 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:59:35.831 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:36.011 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-09-30T00:59:36.030 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:59:36.080 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:59:36.100 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cephfs. 2023-09-30T00:59:36.119 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../006-python3-cephfs_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:36.158 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:36.478 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-ceph-common. 2023-09-30T00:59:36.498 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:59:36.536 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:36.590 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-09-30T00:59:36.606 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:59:36.644 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:59:36.789 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-prettytable. 2023-09-30T00:59:36.806 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2023-09-30T00:59:36.839 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-09-30T00:59:37.001 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-09-30T00:59:37.016 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:59:37.042 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rbd. 2023-09-30T00:59:37.061 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../009-python3-rbd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:37.092 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:59:37.101 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:37.678 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-09-30T00:59:37.698 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-09-30T00:59:37.712 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-09-30T00:59:37.729 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-09-30T00:59:37.736 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-30T00:59:37.770 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-30T00:59:38.081 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-09-30T00:59:38.098 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2023-09-30T00:59:38.139 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-09-30T00:59:38.140 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package librgw2. 2023-09-30T00:59:38.159 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../019-librgw2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:38.224 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:38.434 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-09-30T00:59:38.454 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-09-30T00:59:38.492 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-30T00:59:38.837 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-09-30T00:59:38.857 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-09-30T00:59:38.895 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-30T00:59:39.043 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-rgw. 2023-09-30T00:59:39.059 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../020-python3-rgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:39.101 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:39.223 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-09-30T00:59:39.240 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-09-30T00:59:39.282 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-30T00:59:39.479 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package liboath0. 2023-09-30T00:59:39.499 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2023-09-30T00:59:39.570 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-09-30T00:59:39.687 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-09-30T00:59:39.706 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:59:39.747 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:59:39.915 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libradosstriper1. 2023-09-30T00:59:39.934 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../022-libradosstriper1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:39.973 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:40.291 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-09-30T00:59:40.308 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:59:40.349 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:59:40.493 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-common. 2023-09-30T00:59:40.509 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../023-ceph-common_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:40.551 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:40.700 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-09-30T00:59:40.716 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:59:40.758 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:59:41.120 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-09-30T00:59:41.136 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-09-30T00:59:41.170 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-30T00:59:41.539 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package librgw2. 2023-09-30T00:59:41.556 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../019-librgw2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:41.597 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:42.319 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rgw. 2023-09-30T00:59:42.336 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../020-python3-rgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:42.369 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:42.739 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liboath0. 2023-09-30T00:59:42.756 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2023-09-30T00:59:42.797 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-09-30T00:59:43.100 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libradosstriper1. 2023-09-30T00:59:43.116 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../022-libradosstriper1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:43.158 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:43.240 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-base. 2023-09-30T00:59:43.258 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../024-ceph-base_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:43.300 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:43.570 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-common. 2023-09-30T00:59:43.587 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../023-ceph-common_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:43.620 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:44.329 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-cryptography. 2023-09-30T00:59:44.347 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-09-30T00:59:44.378 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-30T00:59:44.824 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-openssl. 2023-09-30T00:59:44.842 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2023-09-30T00:59:44.882 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-09-30T00:59:45.176 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-09-30T00:59:45.194 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:59:45.234 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:45.681 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-bcrypt. 2023-09-30T00:59:45.697 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-09-30T00:59:45.739 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-30T00:59:46.058 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-cherrypy3. 2023-09-30T00:59:46.078 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-6_all.deb ... 2023-09-30T00:59:46.117 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-09-30T00:59:46.258 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-base. 2023-09-30T00:59:46.278 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../024-ceph-base_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:46.327 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:46.503 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-natsort. 2023-09-30T00:59:46.519 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../030-python3-natsort_7.0.1-1_all.deb ... 2023-09-30T00:59:46.561 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-09-30T00:59:46.898 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-jwt. 2023-09-30T00:59:46.915 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../031-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-09-30T00:59:46.956 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-30T00:59:47.313 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cryptography. 2023-09-30T00:59:47.333 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-09-30T00:59:47.354 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-logutils. 2023-09-30T00:59:47.368 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-7_all.deb ... 2023-09-30T00:59:47.371 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-30T00:59:47.409 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-09-30T00:59:47.745 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-markupsafe. 2023-09-30T00:59:47.761 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../033-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-09-30T00:59:47.783 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-openssl. 2023-09-30T00:59:47.802 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2023-09-30T00:59:47.841 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-09-30T00:59:47.862 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-09-30T00:59:48.127 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-09-30T00:59:48.144 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:59:48.185 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:48.190 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-mako. 2023-09-30T00:59:48.210 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../034-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-09-30T00:59:48.248 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-30T00:59:48.591 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-bcrypt. 2023-09-30T00:59:48.601 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-simplegeneric. 2023-09-30T00:59:48.608 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-09-30T00:59:48.621 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-3_all.deb ... 2023-09-30T00:59:48.649 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-30T00:59:48.684 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-09-30T00:59:48.952 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cherrypy3. 2023-09-30T00:59:48.969 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-6_all.deb ... 2023-09-30T00:59:49.011 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-09-30T00:59:49.021 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-singledispatch. 2023-09-30T00:59:49.037 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-09-30T00:59:49.079 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-09-30T00:59:49.339 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-natsort. 2023-09-30T00:59:49.358 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../030-python3-natsort_7.0.1-1_all.deb ... 2023-09-30T00:59:49.390 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-webob. 2023-09-30T00:59:49.397 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-09-30T00:59:49.407 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../037-python3-webob_1%3a1.8.5-2_all.deb ... 2023-09-30T00:59:49.448 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-09-30T00:59:49.709 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-jwt. 2023-09-30T00:59:49.725 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../031-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-09-30T00:59:49.767 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-30T00:59:49.826 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-soupsieve. 2023-09-30T00:59:49.843 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../038-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-09-30T00:59:49.884 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-30T00:59:50.078 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-logutils. 2023-09-30T00:59:50.098 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-7_all.deb ... 2023-09-30T00:59:50.136 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-09-30T00:59:50.221 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-bs4. 2023-09-30T00:59:50.236 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../039-python3-bs4_4.8.2-1_all.deb ... 2023-09-30T00:59:50.303 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-09-30T00:59:50.439 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-markupsafe. 2023-09-30T00:59:50.456 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../033-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-09-30T00:59:50.514 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-09-30T00:59:50.698 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-waitress. 2023-09-30T00:59:50.716 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../040-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-09-30T00:59:50.759 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-30T00:59:50.809 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-mako. 2023-09-30T00:59:50.825 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../034-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-09-30T00:59:50.867 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-30T00:59:51.137 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-tempita. 2023-09-30T00:59:51.157 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-6_all.deb ... 2023-09-30T00:59:51.171 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-simplegeneric. 2023-09-30T00:59:51.190 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-3_all.deb ... 2023-09-30T00:59:51.201 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-09-30T00:59:51.229 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-09-30T00:59:51.532 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-singledispatch. 2023-09-30T00:59:51.537 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-paste. 2023-09-30T00:59:51.548 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-09-30T00:59:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../042-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-09-30T00:59:51.590 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-09-30T00:59:51.595 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-09-30T00:59:51.926 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-webob. 2023-09-30T00:59:51.931 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-09-30T00:59:51.943 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../037-python3-webob_1%3a1.8.5-2_all.deb ... 2023-09-30T00:59:51.949 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-09-30T00:59:51.984 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-09-30T00:59:52.023 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-09-30T00:59:52.321 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-soupsieve. 2023-09-30T00:59:52.337 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../038-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-09-30T00:59:52.351 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pastedeploy. 2023-09-30T00:59:52.368 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../044-python3-pastedeploy_2.1.0-1_all.deb ... 2023-09-30T00:59:52.370 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-30T00:59:52.409 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-09-30T00:59:52.698 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-bs4. 2023-09-30T00:59:52.715 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../039-python3-bs4_4.8.2-1_all.deb ... 2023-09-30T00:59:52.754 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-webtest. 2023-09-30T00:59:52.756 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-09-30T00:59:52.771 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../045-python3-webtest_2.0.32-2_all.deb ... 2023-09-30T00:59:52.837 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-09-30T00:59:53.891 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pecan. 2023-09-30T00:59:53.893 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-waitress. 2023-09-30T00:59:53.910 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../046-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-09-30T00:59:53.913 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../040-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-09-30T00:59:53.948 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-09-30T00:59:53.954 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-30T00:59:54.279 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-tempita. 2023-09-30T00:59:54.296 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-6_all.deb ... 2023-09-30T00:59:54.338 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-09-30T00:59:54.353 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-werkzeug. 2023-09-30T00:59:54.369 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../047-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-09-30T00:59:54.411 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-30T00:59:54.665 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-paste. 2023-09-30T00:59:54.683 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../042-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-09-30T00:59:54.723 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-09-30T00:59:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-09-30T00:59:54.791 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:54.857 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:55.044 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-09-30T00:59:55.061 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-09-30T00:59:55.102 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-09-30T00:59:55.250 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr. 2023-09-30T00:59:55.268 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../049-ceph-mgr_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:55.308 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:55.404 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pastedeploy. 2023-09-30T00:59:55.421 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../044-python3-pastedeploy_2.1.0-1_all.deb ... 2023-09-30T00:59:55.463 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-09-30T00:59:55.766 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-webtest. 2023-09-30T00:59:55.782 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../045-python3-webtest_2.0.32-2_all.deb ... 2023-09-30T00:59:55.820 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mon. 2023-09-30T00:59:55.836 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../050-ceph-mon_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:55.842 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-09-30T00:59:55.886 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:56.160 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pecan. 2023-09-30T00:59:56.176 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../046-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-09-30T00:59:56.218 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-09-30T00:59:56.556 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-werkzeug. 2023-09-30T00:59:56.573 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../047-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-09-30T00:59:56.606 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-30T00:59:56.959 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-09-30T00:59:56.976 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:57.026 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:57.115 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-osd. 2023-09-30T00:59:57.132 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../051-ceph-osd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:57.173 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:57.387 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr. 2023-09-30T00:59:57.404 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../049-ceph-mgr_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:57.445 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:57.940 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mon. 2023-09-30T00:59:57.957 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../050-ceph-mon_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:57.998 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:59:59.061 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-osd. 2023-09-30T00:59:59.077 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../051-ceph-osd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:59:59.110 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:00.126 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph. 2023-09-30T01:00:00.143 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../052-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:00.184 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:00.512 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-fuse. 2023-09-30T01:00:00.530 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../053-ceph-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:00.586 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:01.182 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mds. 2023-09-30T01:00:01.197 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../054-ceph-mds_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:01.239 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:01.801 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package cephadm. 2023-09-30T01:00:01.819 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../055-cephadm_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:01.859 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:01.980 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph. 2023-09-30T01:00:01.997 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../052-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:02.038 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:02.195 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-jinja2. 2023-09-30T01:00:02.211 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2_all.deb ... 2023-09-30T01:00:02.270 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-09-30T01:00:02.357 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-fuse. 2023-09-30T01:00:02.375 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../053-ceph-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:02.432 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:02.589 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-09-30T01:00:02.609 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T01:00:02.639 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:02.844 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mds. 2023-09-30T01:00:02.862 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../054-ceph-mds_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:02.902 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:02.976 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-repoze.lru. 2023-09-30T01:00:02.994 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-2_all.deb ... 2023-09-30T01:00:03.025 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-09-30T01:00:03.270 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libjs-underscore. 2023-09-30T01:00:03.290 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../059-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-09-30T01:00:03.328 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-30T01:00:03.430 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package cephadm. 2023-09-30T01:00:03.447 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../055-cephadm_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:03.488 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-09-30T01:00:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../060-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-09-30T01:00:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-30T01:00:03.799 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-jinja2. 2023-09-30T01:00:03.815 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2_all.deb ... 2023-09-30T01:00:03.898 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-09-30T01:00:04.075 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-routes. 2023-09-30T01:00:04.091 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../061-python3-routes_2.4.1-2_all.deb ... 2023-09-30T01:00:04.132 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-09-30T01:00:04.227 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-09-30T01:00:04.243 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T01:00:04.293 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:04.519 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-09-30T01:00:04.538 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../062-ceph-mgr-dashboard_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T01:00:04.585 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:04.638 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-repoze.lru. 2023-09-30T01:00:04.655 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-2_all.deb ... 2023-09-30T01:00:04.696 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-09-30T01:00:04.932 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libjs-underscore. 2023-09-30T01:00:04.950 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../059-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-09-30T01:00:04.990 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-30T01:00:05.147 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-decorator. 2023-09-30T01:00:05.164 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../063-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-09-30T01:00:05.205 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-09-30T01:00:05.326 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-09-30T01:00:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../060-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-09-30T01:00:05.384 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-30T01:00:05.533 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-09-30T01:00:05.550 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../064-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-09-30T01:00:05.591 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-30T01:00:05.703 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-routes. 2023-09-30T01:00:05.722 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../061-python3-routes_2.4.1-2_all.deb ... 2023-09-30T01:00:05.761 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-09-30T01:00:05.969 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-scipy. 2023-09-30T01:00:05.987 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../065-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-09-30T01:00:06.027 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-09-30T01:00:06.089 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-09-30T01:00:06.105 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../062-ceph-mgr-dashboard_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T01:00:06.147 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:06.651 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-decorator. 2023-09-30T01:00:06.670 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../063-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-09-30T01:00:06.709 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-09-30T01:00:07.012 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-09-30T01:00:07.028 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../064-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-09-30T01:00:07.070 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-30T01:00:07.389 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-scipy. 2023-09-30T01:00:07.406 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../065-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-09-30T01:00:07.410 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-09-30T01:00:07.428 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../066-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-09-30T01:00:07.447 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-09-30T01:00:07.460 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-30T01:00:08.097 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-joblib. 2023-09-30T01:00:08.113 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../067-python3-joblib_0.14.0-3_all.deb ... 2023-09-30T01:00:08.188 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-09-30T01:00:08.617 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-sklearn. 2023-09-30T01:00:08.635 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../068-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-09-30T01:00:08.700 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-30T01:00:08.723 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-09-30T01:00:08.743 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../066-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-09-30T01:00:08.781 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-30T01:00:09.618 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-joblib. 2023-09-30T01:00:09.638 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../067-python3-joblib_0.14.0-3_all.deb ... 2023-09-30T01:00:09.676 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-09-30T01:00:09.808 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-09-30T01:00:09.825 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../069-ceph-mgr-diskprediction-local_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T01:00:09.866 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:10.055 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-sklearn. 2023-09-30T01:00:10.072 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../068-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-09-30T01:00:10.113 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-30T01:00:10.838 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-09-30T01:00:10.858 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../069-ceph-mgr-diskprediction-local_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T01:00:10.896 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-cachetools. 2023-09-30T01:00:11.069 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../070-python3-cachetools_4.0.0-1_all.deb ... 2023-09-30T01:00:11.113 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-09-30T01:00:11.464 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pyasn1. 2023-09-30T01:00:11.483 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-09-30T01:00:11.521 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-09-30T01:00:11.868 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-09-30T01:00:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-09-30T01:00:11.924 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-30T01:00:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-cachetools. 2023-09-30T01:00:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../070-python3-cachetools_4.0.0-1_all.deb ... 2023-09-30T01:00:12.082 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-09-30T01:00:12.319 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-rsa. 2023-09-30T01:00:12.336 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../073-python3-rsa_4.0-3_all.deb ... 2023-09-30T01:00:12.376 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pyasn1. 2023-09-30T01:00:12.377 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-rsa (4.0-3) ... 2023-09-30T01:00:12.393 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-09-30T01:00:12.435 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-09-30T01:00:12.713 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-google-auth. 2023-09-30T01:00:12.730 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../074-python3-google-auth_1.5.1-2_all.deb ... 2023-09-30T01:00:12.746 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-09-30T01:00:12.763 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-09-30T01:00:12.771 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-09-30T01:00:12.804 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-30T01:00:13.140 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-rsa. 2023-09-30T01:00:13.149 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-blinker. 2023-09-30T01:00:13.157 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../073-python3-rsa_4.0-3_all.deb ... 2023-09-30T01:00:13.166 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../075-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-09-30T01:00:13.198 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-rsa (4.0-3) ... 2023-09-30T01:00:13.207 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-30T01:00:13.510 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-google-auth. 2023-09-30T01:00:13.518 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-oauthlib. 2023-09-30T01:00:13.528 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../074-python3-google-auth_1.5.1-2_all.deb ... 2023-09-30T01:00:13.538 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../076-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-09-30T01:00:13.577 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-30T01:00:13.577 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-09-30T01:00:13.896 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-blinker. 2023-09-30T01:00:13.915 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../075-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-09-30T01:00:13.954 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-30T01:00:13.963 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-09-30T01:00:13.982 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../077-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-09-30T01:00:14.021 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-30T01:00:14.257 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-oauthlib. 2023-09-30T01:00:14.278 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../076-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-09-30T01:00:14.307 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-30T01:00:14.357 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-websocket. 2023-09-30T01:00:14.374 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../078-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-09-30T01:00:14.415 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-09-30T01:00:14.652 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-09-30T01:00:14.671 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../077-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-09-30T01:00:14.710 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-30T01:00:14.802 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-kubernetes. 2023-09-30T01:00:14.820 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../079-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-09-30T01:00:14.860 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-09-30T01:00:14.955 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-websocket. 2023-09-30T01:00:14.976 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../078-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-09-30T01:00:15.004 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-09-30T01:00:15.308 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-kubernetes. 2023-09-30T01:00:15.324 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../079-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-09-30T01:00:15.365 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-09-30T01:00:15.490 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-09-30T01:00:15.511 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../080-ceph-mgr-k8sevents_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T01:00:15.581 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:15.962 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-09-30T01:00:15.980 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../080-ceph-mgr-k8sevents_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T01:00:16.011 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libonig5:amd64. 2023-09-30T01:00:16.020 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:16.028 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../081-libonig5_6.9.4-1_amd64.deb ... 2023-09-30T01:00:16.094 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-09-30T01:00:16.450 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libonig5:amd64. 2023-09-30T01:00:16.468 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../081-libonig5_6.9.4-1_amd64.deb ... 2023-09-30T01:00:16.489 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libjq1:amd64. 2023-09-30T01:00:16.507 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../082-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-30T01:00:16.508 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-09-30T01:00:16.547 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-30T01:00:16.850 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package jq. 2023-09-30T01:00:16.868 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../083-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-30T01:00:16.870 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libjq1:amd64. 2023-09-30T01:00:16.888 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../082-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-30T01:00:16.928 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-30T01:00:16.941 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-09-30T01:00:17.219 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package socat. 2023-09-30T01:00:17.231 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package jq. 2023-09-30T01:00:17.236 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../084-socat_1.7.3.3-2_amd64.deb ... 2023-09-30T01:00:17.250 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../083-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-30T01:00:17.277 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking socat (1.7.3.3-2) ... 2023-09-30T01:00:17.289 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-09-30T01:00:17.542 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package socat. 2023-09-30T01:00:17.559 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../084-socat_1.7.3.3-2_amd64.deb ... 2023-09-30T01:00:17.600 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking socat (1.7.3.3-2) ... 2023-09-30T01:00:17.631 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package xmlstarlet. 2023-09-30T01:00:17.648 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-09-30T01:00:17.689 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-09-30T01:00:17.912 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package xmlstarlet. 2023-09-30T01:00:17.929 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-09-30T01:00:17.970 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-09-30T01:00:18.093 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-test. 2023-09-30T01:00:18.114 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../086-ceph-test_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:18.151 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:18.333 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-test. 2023-09-30T01:00:18.353 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../086-ceph-test_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:18.391 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:23.395 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package ceph-volume. 2023-09-30T01:00:23.413 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../087-ceph-volume_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T01:00:23.453 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:23.549 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-volume. 2023-09-30T01:00:23.570 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../087-ceph-volume_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T01:00:23.607 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:23.800 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package fonts-mathjax. 2023-09-30T01:00:23.818 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../088-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-30T01:00:23.858 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-30T01:00:23.959 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package fonts-mathjax. 2023-09-30T01:00:23.976 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../088-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-30T01:00:24.017 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-30T01:00:24.337 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libcephfs-dev. 2023-09-30T01:00:24.356 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../089-libcephfs-dev_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:24.395 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:24.562 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libcephfs-dev. 2023-09-30T01:00:24.580 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../089-libcephfs-dev_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:24.663 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:24.773 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package nvme-cli. 2023-09-30T01:00:24.794 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-09-30T01:00:24.849 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-09-30T01:00:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package nvme-cli. 2023-09-30T01:00:25.100 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-09-30T01:00:25.179 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-09-30T01:00:25.202 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package libjs-mathjax. 2023-09-30T01:00:25.219 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../091-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-30T01:00:25.260 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-30T01:00:25.528 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libjs-mathjax. 2023-09-30T01:00:25.547 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../091-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-30T01:00:25.586 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-30T01:00:26.858 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python-natsort-doc. 2023-09-30T01:00:26.876 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../092-python-natsort-doc_7.0.1-1_all.deb ... 2023-09-30T01:00:26.915 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-09-30T01:00:27.243 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-atomicwrites. 2023-09-30T01:00:27.261 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../093-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-09-30T01:00:27.301 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-09-30T01:00:27.349 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python-natsort-doc. 2023-09-30T01:00:27.367 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../092-python-natsort-doc_7.0.1-1_all.deb ... 2023-09-30T01:00:27.407 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-09-30T01:00:27.589 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-attr. 2023-09-30T01:00:27.608 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../094-python3-attr_19.3.0-2_all.deb ... 2023-09-30T01:00:27.655 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-09-30T01:00:27.743 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-atomicwrites. 2023-09-30T01:00:27.766 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../093-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-09-30T01:00:27.826 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-09-30T01:00:27.975 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-webencodings. 2023-09-30T01:00:27.995 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../095-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-09-30T01:00:28.033 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-30T01:00:28.179 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-attr. 2023-09-30T01:00:28.197 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../094-python3-attr_19.3.0-2_all.deb ... 2023-09-30T01:00:28.237 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-09-30T01:00:28.336 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-html5lib. 2023-09-30T01:00:28.354 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../096-python3-html5lib_1.0.1-2_all.deb ... 2023-09-30T01:00:28.394 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-09-30T01:00:28.582 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-webencodings. 2023-09-30T01:00:28.599 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../095-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-09-30T01:00:28.656 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-30T01:00:28.713 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-09-30T01:00:28.734 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../097-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-09-30T01:00:28.771 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-30T01:00:28.976 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-html5lib. 2023-09-30T01:00:28.996 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../096-python3-html5lib_1.0.1-2_all.deb ... 2023-09-30T01:00:29.034 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-09-30T01:00:29.275 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-packaging. 2023-09-30T01:00:29.292 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../098-python3-packaging_20.3-1_all.deb ... 2023-09-30T01:00:29.333 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-packaging (20.3-1) ... 2023-09-30T01:00:29.420 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-09-30T01:00:29.440 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../097-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-09-30T01:00:29.487 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-30T01:00:29.636 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pastescript. 2023-09-30T01:00:29.653 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2023-09-30T01:00:29.694 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-09-30T01:00:30.006 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pluggy. 2023-09-30T01:00:30.007 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-packaging. 2023-09-30T01:00:30.026 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2023-09-30T01:00:30.027 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../098-python3-packaging_20.3-1_all.deb ... 2023-09-30T01:00:30.063 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-09-30T01:00:30.064 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-packaging (20.3-1) ... 2023-09-30T01:00:30.400 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-psutil. 2023-09-30T01:00:30.409 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pastescript. 2023-09-30T01:00:30.417 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-09-30T01:00:30.428 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2023-09-30T01:00:30.458 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-09-30T01:00:30.468 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-09-30T01:00:30.778 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-py. 2023-09-30T01:00:30.794 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-09-30T01:00:30.836 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-09-30T01:00:30.837 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pluggy. 2023-09-30T01:00:30.854 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2023-09-30T01:00:30.895 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-09-30T01:00:31.172 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pyinotify. 2023-09-30T01:00:31.190 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-09-30T01:00:31.230 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-30T01:00:31.240 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-psutil. 2023-09-30T01:00:31.257 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-09-30T01:00:31.298 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-09-30T01:00:31.566 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-wcwidth. 2023-09-30T01:00:31.585 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-09-30T01:00:31.624 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-30T01:00:31.676 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-py. 2023-09-30T01:00:31.693 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-09-30T01:00:31.734 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-09-30T01:00:31.927 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-pytest. 2023-09-30T01:00:31.945 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2023-09-30T01:00:31.985 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-09-30T01:00:32.070 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pyinotify. 2023-09-30T01:00:32.091 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-09-30T01:00:32.128 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-30T01:00:32.305 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package python3-simplejson. 2023-09-30T01:00:32.326 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-09-30T01:00:32.363 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-30T01:00:32.490 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-wcwidth. 2023-09-30T01:00:32.508 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-09-30T01:00:32.548 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-30T01:00:32.666 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-09-30T01:00:32.684 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-09-30T01:00:32.724 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-30T01:00:32.885 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pytest. 2023-09-30T01:00:32.902 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2023-09-30T01:00:32.943 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-09-30T01:00:33.321 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-simplejson. 2023-09-30T01:00:33.338 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-09-30T01:00:33.379 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-30T01:00:33.445 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package radosgw. 2023-09-30T01:00:33.462 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../108-radosgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:33.503 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:33.682 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-09-30T01:00:33.702 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-09-30T01:00:33.731 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-30T01:00:34.553 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package radosgw. 2023-09-30T01:00:34.570 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../108-radosgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:34.611 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:35.240 INFO:teuthology.orchestra.run.smithi050.stdout:Selecting previously unselected package rbd-fuse. 2023-09-30T01:00:35.257 INFO:teuthology.orchestra.run.smithi050.stdout:Preparing to unpack .../109-rbd-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:35.298 INFO:teuthology.orchestra.run.smithi050.stdout:Unpacking rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:35.634 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-attr (19.3.0-2) ... 2023-09-30T01:00:35.928 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-09-30T01:00:36.029 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-30T01:00:36.147 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-09-30T01:00:36.349 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:36.523 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package rbd-fuse. 2023-09-30T01:00:36.540 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../109-rbd-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T01:00:36.580 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:36.911 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-attr (19.3.0-2) ... 2023-09-30T01:00:36.994 INFO:teuthology.orchestra.run.smithi050.stdout:Adding system user cephadm....done 2023-09-30T01:00:37.155 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-30T01:00:37.273 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-30T01:00:37.280 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-09-30T01:00:37.415 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-30T01:00:37.574 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-09-30T01:00:37.620 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-09-30T01:00:37.793 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:37.913 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-30T01:00:38.039 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-09-30T01:00:38.181 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-09-30T01:00:38.462 INFO:teuthology.orchestra.run.smithi037.stdout:Adding system user cephadm....done 2023-09-30T01:00:38.500 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-09-30T01:00:38.648 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-30T01:00:38.775 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-30T01:00:38.927 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-30T01:00:39.169 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-09-30T01:00:39.211 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-09-30T01:00:39.478 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-30T01:00:39.504 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-30T01:00:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-09-30T01:00:39.780 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-09-30T01:00:40.130 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-30T01:00:40.415 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:40.476 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-09-30T01:00:40.690 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-30T01:00:40.850 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-09-30T01:00:40.878 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-30T01:00:41.135 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-30T01:00:41.172 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-09-30T01:00:41.436 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-30T01:00:41.465 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-30T01:00:41.562 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-09-30T01:00:41.757 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-30T01:00:41.938 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-09-30T01:00:42.074 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:42.222 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-09-30T01:00:42.392 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-30T01:00:42.527 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-30T01:00:42.545 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-09-30T01:00:42.829 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-30T01:00:42.870 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-09-30T01:00:43.122 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-30T01:00:43.146 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-09-30T01:00:43.267 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-09-30T01:00:43.481 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-09-30T01:00:43.658 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-09-30T01:00:43.757 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-packaging (20.3-1) ... 2023-09-30T01:00:43.950 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-09-30T01:00:44.042 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-09-30T01:00:44.169 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-30T01:00:44.226 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-30T01:00:44.293 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-30T01:00:44.578 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-09-30T01:00:44.580 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-09-30T01:00:44.887 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-09-30T01:00:45.006 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-30T01:00:45.271 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-09-30T01:00:45.374 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-30T01:00:45.556 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-packaging (20.3-1) ... 2023-09-30T01:00:45.735 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up socat (1.7.3.3-2) ... 2023-09-30T01:00:45.857 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-09-30T01:00:45.860 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:45.983 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-30T01:00:46.125 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-30T01:00:46.187 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T01:00:46.296 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-09-30T01:00:46.429 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-09-30T01:00:46.648 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-30T01:00:46.774 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-09-30T01:00:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-30T01:00:46.908 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-09-30T01:00:47.201 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-30T01:00:47.223 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-30T01:00:47.544 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T01:00:47.583 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up socat (1.7.3.3-2) ... 2023-09-30T01:00:47.645 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-30T01:00:47.742 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:47.746 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-09-30T01:00:48.048 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-09-30T01:00:48.060 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T01:00:48.203 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-09-30T01:00:48.324 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-09-30T01:00:48.605 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-30T01:00:48.658 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-30T01:00:48.731 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-09-30T01:00:48.784 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-09-30T01:00:48.890 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-09-30T01:00:49.060 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-09-30T01:00:49.201 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-30T01:00:49.353 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-30T01:00:49.512 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T01:00:49.637 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-09-30T01:00:49.638 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-30T01:00:49.798 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-09-30T01:00:50.017 INFO:teuthology.orchestra.run.smithi050.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-09-30T01:00:50.124 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-09-30T01:00:50.274 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-09-30T01:00:50.396 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-30T01:00:50.417 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-09-30T01:00:50.690 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-09-30T01:00:50.718 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-30T01:00:50.872 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-09-30T01:00:51.041 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-30T01:00:51.167 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-30T01:00:51.173 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-09-30T01:00:51.318 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-09-30T01:00:51.483 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-30T01:00:51.653 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-30T01:00:51.792 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-09-30T01:00:51.921 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librados2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:52.042 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-09-30T01:00:52.148 INFO:teuthology.orchestra.run.smithi037.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-09-30T01:00:52.428 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-09-30T01:00:52.596 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-30T01:00:52.932 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-09-30T01:00:53.309 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-30T01:00:53.459 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-30T01:00:53.573 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:53.585 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-09-30T01:00:53.699 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-30T01:00:53.825 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-09-30T01:00:53.970 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-30T01:00:54.213 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-09-30T01:00:54.263 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up librados2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:54.389 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-09-30T01:00:54.497 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-09-30T01:00:54.782 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T01:00:54.908 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-09-30T01:00:55.251 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:55.377 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-30T01:00:55.712 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:55.813 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-30T01:00:55.936 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:56.070 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-30T01:00:56.123 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-30T01:00:56.238 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-09-30T01:00:56.615 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-09-30T01:00:56.924 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-09-30T01:00:57.187 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librbd1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:57.255 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T01:00:57.288 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-30T01:00:57.384 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-09-30T01:00:57.397 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-09-30T01:00:57.524 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-09-30T01:00:57.650 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rsa (4.0-3) ... 2023-09-30T01:00:57.719 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:57.878 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-30T01:00:57.959 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-30T01:00:58.085 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-09-30T01:00:58.205 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:58.364 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-30T01:00:58.403 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:58.504 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:58.699 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-30T01:00:58.719 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-30T01:00:58.719 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-30T01:00:59.181 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:59.392 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-30T01:00:59.660 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-09-30T01:00:59.771 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up librbd1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:00:59.897 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-30T01:00:59.962 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-09-30T01:01:00.048 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-09-30T01:01:00.165 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-09-30T01:01:00.306 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:00.308 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-rsa (4.0-3) ... 2023-09-30T01:01:00.452 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:00.567 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:00.601 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-30T01:01:00.668 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:00.760 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-09-30T01:01:00.768 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-routes (2.4.1-2) ... 2023-09-30T01:01:01.061 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:01.071 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-09-30T01:01:01.187 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:01.380 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-30T01:01:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-30T01:01:01.506 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:01.633 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-09-30T01:01:01.947 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:02.192 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-30T01:01:02.485 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-09-30T01:01:02.577 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:02.795 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-09-30T01:01:03.139 INFO:teuthology.orchestra.run.smithi050.stdout:Adding group ceph....done 2023-09-30T01:01:03.164 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:03.323 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:03.474 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:03.499 INFO:teuthology.orchestra.run.smithi050.stdout:Adding system user ceph....done 2023-09-30T01:01:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:03.675 INFO:teuthology.orchestra.run.smithi050.stdout:Setting system user ceph properties....done 2023-09-30T01:01:03.701 INFO:teuthology.orchestra.run.smithi050.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-09-30T01:01:03.768 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-routes (2.4.1-2) ... 2023-09-30T01:01:03.854 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-09-30T01:01:04.086 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-09-30T01:01:04.175 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-09-30T01:01:04.396 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:04.522 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:04.630 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:04.681 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-09-30T01:01:04.756 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:05.036 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-30T01:01:05.036 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-30T01:01:05.610 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:05.691 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:06.123 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-09-30T01:01:06.238 INFO:teuthology.orchestra.run.smithi037.stdout:Adding group ceph....done 2023-09-30T01:01:06.585 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:06.639 INFO:teuthology.orchestra.run.smithi037.stdout:Adding system user ceph....done 2023-09-30T01:01:06.822 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-30T01:01:06.822 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-30T01:01:06.849 INFO:teuthology.orchestra.run.smithi037.stdout:Setting system user ceph properties....done 2023-09-30T01:01:06.873 INFO:teuthology.orchestra.run.smithi037.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-09-30T01:01:07.021 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-09-30T01:01:07.270 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:07.335 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-09-30T01:01:07.494 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-30T01:01:07.495 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-30T01:01:07.803 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:07.955 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:08.004 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:08.286 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-30T01:01:08.286 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-30T01:01:08.460 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-30T01:01:08.460 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-30T01:01:08.915 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:08.948 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:09.032 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:09.158 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:09.362 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-09-30T01:01:09.391 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-30T01:01:09.391 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-30T01:01:09.819 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:09.843 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:09.994 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:10.082 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-30T01:01:10.082 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-30T01:01:10.120 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:10.271 INFO:teuthology.orchestra.run.smithi050.stdout:Setting up ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:10.391 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-09-30T01:01:10.528 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-09-30T01:01:10.546 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:10.777 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-30T01:01:10.777 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-30T01:01:10.821 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-09-30T01:01:11.414 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:11.917 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-30T01:01:11.917 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-30T01:01:12.374 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:12.525 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:12.670 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:12.918 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-30T01:01:12.918 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-30T01:01:13.369 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:13.545 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:13.733 INFO:teuthology.orchestra.run.smithi050.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-09-30T01:01:13.864 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T01:01:13.983 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-09-30T01:01:14.161 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-09-30T01:01:14.341 INFO:teuthology.orchestra.run.smithi050.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-30T01:01:14.343 DEBUG:teuthology.parallel:result is None 2023-09-30T01:01:14.454 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-09-30T01:01:17.349 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-09-30T01:01:17.951 INFO:teuthology.orchestra.run.smithi037.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-30T01:01:17.954 DEBUG:teuthology.parallel:result is None 2023-09-30T01:01:17.954 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-30T01:01:18.194 DEBUG:teuthology.orchestra.run.smithi037:> dpkg-query -W -f '${Version}' ceph 2023-09-30T01:01:18.219 INFO:teuthology.orchestra.run.smithi037.stdout:17.2.6-1165-g93dce5e1-1focal 2023-09-30T01:01:18.219 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1165-g93dce5e1-1focal 2023-09-30T01:01:18.220 INFO:teuthology.task.install:The correct ceph version 17.2.6-1165-g93dce5e1-1focal is installed. 2023-09-30T01:01:18.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-30T01:01:18.505 DEBUG:teuthology.orchestra.run.smithi050:> dpkg-query -W -f '${Version}' ceph 2023-09-30T01:01:18.529 INFO:teuthology.orchestra.run.smithi050.stdout:17.2.6-1165-g93dce5e1-1focal 2023-09-30T01:01:18.529 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1165-g93dce5e1-1focal 2023-09-30T01:01:18.529 INFO:teuthology.task.install:The correct ceph version 17.2.6-1165-g93dce5e1-1focal is installed. 2023-09-30T01:01:18.530 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-30T01:01:18.530 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:18.530 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T01:01:18.545 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:18.545 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T01:01:18.587 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-30T01:01:18.587 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:18.588 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T01:01:18.602 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T01:01:18.660 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:18.660 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T01:01:18.676 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T01:01:18.686 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-30T01:01:18.686 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:18.686 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T01:01:18.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T01:01:18.758 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:18.758 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T01:01:18.774 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T01:01:18.785 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-30T01:01:18.786 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:18.786 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T01:01:18.802 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T01:01:18.860 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:18.860 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T01:01:18.875 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T01:01:18.931 INFO:teuthology.run_tasks:Running task ceph... 2023-09-30T01:01:19.005 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-09-30T01:01:19.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /var/log/ceph 2023-09-30T01:01:19.008 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /var/log/ceph 2023-09-30T01:01:19.024 INFO:tasks.ceph:Disabling ceph logrotate... 2023-09-30T01:01:19.024 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-30T01:01:19.070 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-30T01:01:19.086 INFO:tasks.ceph:Creating extra log directories... 2023-09-30T01:01:19.086 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-30T01:01:19.088 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-30T01:01:19.157 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-09-30T01:01:19.157 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd inject bad map crc probability': 0.1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 2, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2023-09-30T01:01:19.158 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821', 'branch': 'wip-yuri4-testing-2023-09-29-0928-quincy', 'description': 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7406821', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri4-testing-2023-09-29-0928-quincy'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd inject bad map crc probability': 0.1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 2, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'mon_bind_msgr2': False, 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri4-testing-2023-09-29-0928-quincy', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 9810, 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri4-testing-2023-09-29-0928-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481', 'targets': {'smithi037.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi050.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd inject bad map crc probability': 0.1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 2, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'mon_bind_msgr2': False, 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481', 'cluster': 'ceph'}}, {'exec': {'mon.a': ['while ! ceph balancer status ; do sleep 1 ; done', 'ceph balancer mode crush-compat', 'ceph balancer on']}}, {'thrashosds': {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache min_read_recency_for_promote 0', 'sudo ceph osd pool set cache min_write_recency_for_promote 0']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'write': 100}, 'ops': 4000, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-09-29_19:38:56', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-09-30T01:01:19.158 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-30T01:01:19.161 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-30T01:01:19.169 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/run/ceph 2023-09-30T01:01:19.171 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/run/ceph 2023-09-30T01:01:19.188 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:19.188 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2023-09-30T01:01:19.231 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-30T01:01:19.231 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2023-09-30T01:01:19.239 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-30T01:01:19.239 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T01:01:19.239 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 358 Links: 1 2023-09-30T01:01:19.239 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T01:01:19.239 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2023-09-30 00:57:39.346864606 +0000 2023-09-30T01:01:19.239 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2023-09-30 00:57:39.070870957 +0000 2023-09-30T01:01:19.239 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2023-09-30 00:57:39.070870957 +0000 2023-09-30T01:01:19.239 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: - 2023-09-30T01:01:19.240 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-30T01:01:19.252 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2023-09-30T01:01:19.253 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2023-09-30T01:01:19.253 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000407555 s, 1.3 MB/s 2023-09-30T01:01:19.254 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-30T01:01:19.305 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2023-09-30T01:01:19.314 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-30T01:01:19.314 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T01:01:19.314 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 367 Links: 1 2023-09-30T01:01:19.315 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T01:01:19.315 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2023-09-30 00:57:39.898851904 +0000 2023-09-30T01:01:19.315 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2023-09-30 00:57:39.606858623 +0000 2023-09-30T01:01:19.315 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2023-09-30 00:57:39.606858623 +0000 2023-09-30T01:01:19.315 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: - 2023-09-30T01:01:19.316 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-30T01:01:19.330 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2023-09-30T01:01:19.331 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2023-09-30T01:01:19.331 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000340704 s, 1.5 MB/s 2023-09-30T01:01:19.332 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-30T01:01:19.382 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2023-09-30T01:01:19.389 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-30T01:01:19.390 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T01:01:19.390 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 373 Links: 1 2023-09-30T01:01:19.390 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T01:01:19.390 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2023-09-30 00:57:40.134846472 +0000 2023-09-30T01:01:19.390 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2023-09-30 00:57:40.130846564 +0000 2023-09-30T01:01:19.390 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2023-09-30 00:57:40.130846564 +0000 2023-09-30T01:01:19.390 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: - 2023-09-30T01:01:19.391 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-30T01:01:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2023-09-30T01:01:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2023-09-30T01:01:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000327773 s, 1.6 MB/s 2023-09-30T01:01:19.407 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-30T01:01:19.457 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2023-09-30T01:01:19.465 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-30T01:01:19.465 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T01:01:19.465 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 378 Links: 1 2023-09-30T01:01:19.465 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T01:01:19.465 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2023-09-30 00:57:40.970827233 +0000 2023-09-30T01:01:19.465 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2023-09-30 00:57:40.682833861 +0000 2023-09-30T01:01:19.466 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2023-09-30 00:57:40.682833861 +0000 2023-09-30T01:01:19.466 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: - 2023-09-30T01:01:19.466 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-30T01:01:19.477 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2023-09-30T01:01:19.477 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2023-09-30T01:01:19.477 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000412455 s, 1.2 MB/s 2023-09-30T01:01:19.479 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-30T01:01:19.529 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T01:01:19.529 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:19.530 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2023-09-30T01:01:19.537 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-30T01:01:19.537 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2023-09-30T01:01:19.545 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-30T01:01:19.545 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T01:01:19.545 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 359 Links: 1 2023-09-30T01:01:19.545 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T01:01:19.545 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-09-30 00:57:38.951407333 +0000 2023-09-30T01:01:19.545 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-09-30 00:57:38.655414145 +0000 2023-09-30T01:01:19.545 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-09-30 00:57:38.655414145 +0000 2023-09-30T01:01:19.545 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2023-09-30T01:01:19.546 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-30T01:01:19.558 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-09-30T01:01:19.558 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-09-30T01:01:19.559 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000300794 s, 1.7 MB/s 2023-09-30T01:01:19.560 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-30T01:01:19.568 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2023-09-30T01:01:19.619 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-30T01:01:19.619 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T01:01:19.619 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 364 Links: 1 2023-09-30T01:01:19.619 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T01:01:19.619 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-09-30 00:57:39.459395643 +0000 2023-09-30T01:01:19.619 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-09-30 00:57:39.163402454 +0000 2023-09-30T01:01:19.619 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-09-30 00:57:39.163402454 +0000 2023-09-30T01:01:19.619 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2023-09-30T01:01:19.620 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-30T01:01:19.631 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-09-30T01:01:19.631 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-09-30T01:01:19.632 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.00029168 s, 1.8 MB/s 2023-09-30T01:01:19.632 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-30T01:01:19.641 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2023-09-30T01:01:19.649 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-30T01:01:19.649 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T01:01:19.650 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 373 Links: 1 2023-09-30T01:01:19.650 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T01:01:19.650 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-09-30 00:57:39.947384412 +0000 2023-09-30T01:01:19.650 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-09-30 00:57:39.647391316 +0000 2023-09-30T01:01:19.650 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-09-30 00:57:39.647391316 +0000 2023-09-30T01:01:19.650 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2023-09-30T01:01:19.650 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-30T01:01:19.706 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-09-30T01:01:19.706 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-09-30T01:01:19.706 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.00029449 s, 1.7 MB/s 2023-09-30T01:01:19.707 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-30T01:01:19.716 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2023-09-30T01:01:19.721 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-30T01:01:19.721 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T01:01:19.722 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 6h/6d Inode: 378 Links: 1 2023-09-30T01:01:19.722 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T01:01:19.722 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2023-09-30 00:57:40.455372723 +0000 2023-09-30T01:01:19.722 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2023-09-30 00:57:40.183378982 +0000 2023-09-30T01:01:19.722 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2023-09-30 00:57:40.183378982 +0000 2023-09-30T01:01:19.722 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2023-09-30T01:01:19.722 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-30T01:01:19.778 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2023-09-30T01:01:19.778 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2023-09-30T01:01:19.778 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000299994 s, 1.7 MB/s 2023-09-30T01:01:19.779 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-30T01:01:19.790 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-30T01:01:19.790 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-30T01:01:19.790 INFO:tasks.ceph:Generating config... 2023-09-30T01:01:19.791 INFO:tasks.ceph:[osd] debug monc = 20 2023-09-30T01:01:19.791 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-09-30T01:01:19.791 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2023-09-30T01:01:19.791 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-09-30T01:01:19.792 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-09-30T01:01:19.792 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-09-30T01:01:19.792 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-09-30T01:01:19.792 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-09-30T01:01:19.792 INFO:tasks.ceph:[osd] debug ms = 1 2023-09-30T01:01:19.792 INFO:tasks.ceph:[osd] debug osd = 20 2023-09-30T01:01:19.792 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-09-30T01:01:19.792 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-09-30T01:01:19.792 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-09-30T01:01:19.792 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-09-30T01:01:19.792 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd inject bad map crc probability = 0.1 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd max backfills = 3 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd max scrubs = 2 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-09-30T01:01:19.793 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-09-30T01:01:19.794 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-09-30T01:01:19.794 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-09-30T01:01:19.794 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2023-09-30T01:01:19.794 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-09-30T01:01:19.794 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-09-30T01:01:19.794 INFO:tasks.ceph:[global] mon election default strategy = 1 2023-09-30T01:01:19.794 INFO:tasks.ceph:[global] ms bind msgr2 = False 2023-09-30T01:01:19.794 INFO:tasks.ceph:[global] ms type = async 2023-09-30T01:01:19.794 INFO:tasks.ceph:[global] osd debug inject dispatch delay duration = 0.1 2023-09-30T01:01:19.794 INFO:tasks.ceph:[global] osd debug inject dispatch delay probability = 0.1 2023-09-30T01:01:19.794 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2023-09-30T01:01:19.794 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2023-09-30T01:01:19.795 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-09-30T01:01:19.795 INFO:tasks.ceph:[global] osd_pool_default_size = 2 2023-09-30T01:01:19.795 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-09-30T01:01:19.795 INFO:tasks.ceph:[mgr] debug ms = 1 2023-09-30T01:01:19.795 INFO:tasks.ceph:[mon] debug mon = 20 2023-09-30T01:01:19.795 INFO:tasks.ceph:[mon] debug ms = 1 2023-09-30T01:01:19.795 INFO:tasks.ceph:[mon] debug paxos = 20 2023-09-30T01:01:19.795 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-09-30T01:01:19.795 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-09-30T01:01:19.795 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-09-30T01:01:19.795 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-09-30T01:01:19.795 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-09-30T01:01:19.795 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-09-30T01:01:19.796 INFO:tasks.ceph:Setting up mon.a... 2023-09-30T01:01:19.796 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-09-30T01:01:19.830 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.keyring 2023-09-30T01:01:19.832 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-09-30T01:01:19.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T01:01:19.920 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.37:6789]'), ('mon.c', '[v1:172.21.15.37:6790]'), ('mon.b', '[v1:172.21.15.50:6789]')] 2023-09-30T01:01:19.920 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.37:6789],[v1:172.21.15.37:6790],[v1:172.21.15.50:6789]', 'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd inject bad map crc probability': 0.1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 2, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-09-30T01:01:19.921 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:19.922 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-09-30T01:01:19.926 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.37:6789]' --addv c '[v1:172.21.15.37:6790]' --addv b '[v1:172.21.15.50:6789]' --print /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:19.968 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:19.968 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: generated fsid 43b20d8a-eabf-4c8a-988f-eaf5c9521071 2023-09-30T01:01:19.968 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = octopus 2023-09-30T01:01:19.969 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 0 2023-09-30T01:01:19.969 INFO:teuthology.orchestra.run.smithi037.stdout:fsid 43b20d8a-eabf-4c8a-988f-eaf5c9521071 2023-09-30T01:01:19.969 INFO:teuthology.orchestra.run.smithi037.stdout:last_changed 2023-09-30T01:01:19.967232+0000 2023-09-30T01:01:19.969 INFO:teuthology.orchestra.run.smithi037.stdout:created 2023-09-30T01:01:19.967232+0000 2023-09-30T01:01:19.970 INFO:teuthology.orchestra.run.smithi037.stdout:min_mon_release 15 (octopus) 2023-09-30T01:01:19.970 INFO:teuthology.orchestra.run.smithi037.stdout:election_strategy: 1 2023-09-30T01:01:19.970 INFO:teuthology.orchestra.run.smithi037.stdout:0: v1:172.21.15.37:6789/0 mon.a 2023-09-30T01:01:19.970 INFO:teuthology.orchestra.run.smithi037.stdout:1: v1:172.21.15.50:6789/0 mon.b 2023-09-30T01:01:19.970 INFO:teuthology.orchestra.run.smithi037.stdout:2: v1:172.21.15.37:6790/0 mon.c 2023-09-30T01:01:19.970 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-09-30T01:01:19.971 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-09-30T01:01:20.020 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 43b20d8a-eabf-4c8a-988f-eaf5c9521071... 2023-09-30T01:01:20.021 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-30T01:01:20.023 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-30T01:01:20.058 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2023-09-30T01:01:20.058 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2023-09-30T01:01:20.058 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-30T01:01:20.058 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2023-09-30T01:01:20.058 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-30T01:01:20.059 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2023-09-30T01:01:20.060 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2023-09-30T01:01:20.062 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2023-09-30T01:01:20.062 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2023-09-30T01:01:20.062 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "[v1:172.21.15.37:6789],[v1:172.21.15.37:6790],[v1:172.21.15.50:6789]" 2023-09-30T01:01:20.062 INFO:teuthology.orchestra.run.smithi037.stdout: mon election default strategy = 1 2023-09-30T01:01:20.062 INFO:teuthology.orchestra.run.smithi037.stdout: ms bind msgr2 = False 2023-09-30T01:01:20.062 INFO:teuthology.orchestra.run.smithi037.stdout: ms type = async 2023-09-30T01:01:20.062 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug inject dispatch delay duration = 0.1 2023-09-30T01:01:20.062 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug inject dispatch delay probability = 0.1 2023-09-30T01:01:20.062 INFO:teuthology.orchestra.run.smithi037.stdout: osd_async_recovery_min_cost = 1 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: osd_object_clean_region_max_num_intervals = 1000 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_min_size = 2 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_size = 2 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 43b20d8a-eabf-4c8a-988f-eaf5c9521071 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 120 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.063 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2023-09-30T01:01:20.065 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2023-09-30T01:01:20.065 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.065 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2023-09-30T01:01:20.065 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2023-09-30T01:01:20.065 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2023-09-30T01:01:20.065 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.065 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2023-09-30T01:01:20.065 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2023-09-30T01:01:20.066 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: debug monc = 20 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore compression algorithm = zlib 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore compression mode = aggressive 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore zero block detection = True 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 20 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 20 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 10 2023-09-30T01:01:20.067 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug reject backfill probability = 0.3 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify cached snaps = True 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify missing on start = True 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: osd inject bad map crc probability = 0.1 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: osd max backfills = 3 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: osd max markdown count = 1000 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: osd max scrubs = 2 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock override recovery settings = True 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2023-09-30T01:01:20.068 INFO:teuthology.orchestra.run.smithi037.stdout: osd op queue = debug_random 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout: osd op queue cut off = debug_random 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub min interval = 60 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout: osd shutdown pgref assert = True 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout: osd snap trim sleep = 2 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout: osd_max_pg_log_entries = 3000 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout: osd_min_pg_log_entries = 3000 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2023-09-30T01:01:20.069 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2023-09-30T01:01:20.070 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.071 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout: mon min osdmap epochs = 50 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout: mon osdmap full prune interval = 2 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout: mon osdmap full prune min = 15 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout: mon osdmap full prune txsize = 2 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout: mon scrub interval = 300 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout: paxos service trim min = 10 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2023-09-30T01:01:20.072 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2023-09-30T01:01:20.073 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2023-09-30T01:01:20.073 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2023-09-30T01:01:20.073 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-30T01:01:20.073 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-30T01:01:20.073 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2023-09-30T01:01:20.073 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2023-09-30T01:01:20.073 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2023-09-30T01:01:20.074 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2023-09-30T01:01:20.075 INFO:teuthology.orchestra.run.smithi050.stdout: chdir = "" 2023-09-30T01:01:20.075 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-30T01:01:20.075 INFO:teuthology.orchestra.run.smithi050.stdout: auth supported = cephx 2023-09-30T01:01:20.075 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.075 INFO:teuthology.orchestra.run.smithi050.stdout: filestore xattr use omap = true 2023-09-30T01:01:20.075 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.075 INFO:teuthology.orchestra.run.smithi050.stdout: mon clock drift allowed = 1.000 2023-09-30T01:01:20.075 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.075 INFO:teuthology.orchestra.run.smithi050.stdout: osd crush chooseleaf type = 0 2023-09-30T01:01:20.075 INFO:teuthology.orchestra.run.smithi050.stdout: auth debug = true 2023-09-30T01:01:20.075 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on old message = true 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on bug = true 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: mon pg warn max object skew = 0 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: # disable pg_autoscaler by default for new pools 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default size = 2 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.076 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow primary affinity = true 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow pg remap = true 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on legacy crush tunables = false 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on crush straw calc version zero = false 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on no sortbitwise = false 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on osd down out interval zero = false 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on too few osds = false 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: mon_allow_pool_size_one = true 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.077 INFO:teuthology.orchestra.run.smithi050.stdout: osd default data pool replay window = 5 2023-09-30T01:01:20.078 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.078 INFO:teuthology.orchestra.run.smithi050.stdout: mon allow pool delete = true 2023-09-30T01:01:20.078 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.078 INFO:teuthology.orchestra.run.smithi050.stdout: mon cluster log file level = debug 2023-09-30T01:01:20.078 INFO:teuthology.orchestra.run.smithi050.stdout: debug asserts on shutdown = true 2023-09-30T01:01:20.078 INFO:teuthology.orchestra.run.smithi050.stdout: mon health detail to clog = false 2023-09-30T01:01:20.078 INFO:teuthology.orchestra.run.smithi050.stdout: mon host = "[v1:172.21.15.37:6789],[v1:172.21.15.37:6790],[v1:172.21.15.50:6789]" 2023-09-30T01:01:20.078 INFO:teuthology.orchestra.run.smithi050.stdout: mon election default strategy = 1 2023-09-30T01:01:20.078 INFO:teuthology.orchestra.run.smithi050.stdout: ms bind msgr2 = False 2023-09-30T01:01:20.078 INFO:teuthology.orchestra.run.smithi050.stdout: ms type = async 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug inject dispatch delay duration = 0.1 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug inject dispatch delay probability = 0.1 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: osd_async_recovery_min_cost = 1 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: osd_object_clean_region_max_num_intervals = 1000 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: osd_pool_default_min_size = 2 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: osd_pool_default_size = 2 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = 43b20d8a-eabf-4c8a-988f-eaf5c9521071 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout:[osd] 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: osd journal size = 100 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub load threshold = 5.0 2023-09-30T01:01:20.079 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub max interval = 120 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: osd mclock profile = high_recovery_ops 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: osd recover clone overlap = true 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: osd recovery max chunk = 1048576 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug shutdown = true 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug op order = true 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug verify stray on activate = true 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: osd open classes on start = true 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug pg log writeout = true 2023-09-30T01:01:20.080 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd deep scrub update digest min age = 30 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd map max advance = 10 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: journal zero on create = true 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: filestore ondisk finisher threads = 3 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: filestore apply finisher threads = 3 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: bdev debug aio = true 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug misdirected ops = true 2023-09-30T01:01:20.081 INFO:teuthology.orchestra.run.smithi050.stdout: debug monc = 20 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore block size = 96636764160 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore compression algorithm = zlib 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore compression mode = aggressive 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore fsck on mount = True 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore zero block detection = True 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluefs = 20 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluestore = 20 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: debug osd = 20 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: debug rocksdb = 10 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd backfillfull_ratio = 0.85 2023-09-30T01:01:20.082 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd full ratio = 0.9 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd nearfull ratio = 0.8 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug reject backfill probability = 0.3 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug verify cached snaps = True 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug verify missing on start = True 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd failsafe full ratio = 0.95 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd inject bad map crc probability = 0.1 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd max backfills = 3 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd max markdown count = 1000 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd max scrubs = 2 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd mclock override recovery settings = True 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd objectstore = bluestore 2023-09-30T01:01:20.083 INFO:teuthology.orchestra.run.smithi050.stdout: osd op queue = debug_random 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: osd op queue cut off = debug_random 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub min interval = 60 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: osd shutdown pgref assert = True 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: osd snap trim sleep = 2 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: osd_max_pg_log_entries = 3000 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: osd_min_pg_log_entries = 3000 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr] 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: debug mgr = 20 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2023-09-30T01:01:20.084 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout: mgr/telemetry/nag = false 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout:[mon] 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout: debug paxos = 20 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout: mon data avail warn = 5 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout: mon mgr mkfs grace = 240 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2023-09-30T01:01:20.085 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd reporter subtree level = osd 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd prime pg temp = true 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: auth mon ticket ttl = 660 # 11m 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: auth service ticket ttl = 240 # 4m 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: # don't complain about insecure global_id in the test suite 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: # 1m isn't quite enough 2023-09-30T01:01:20.086 INFO:teuthology.orchestra.run.smithi050.stdout: mon_down_mkfs_grace = 2m 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_filestore_osds = false 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout: mon min osdmap epochs = 50 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout: mon osdmap full prune interval = 2 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout: mon osdmap full prune min = 15 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout: mon osdmap full prune txsize = 2 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout: mon scrub interval = 300 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout: paxos service trim min = 10 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout:[client] 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout: rgw cache enabled = true 2023-09-30T01:01:20.087 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable ops log = true 2023-09-30T01:01:20.088 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable usage log = true 2023-09-30T01:01:20.088 INFO:teuthology.orchestra.run.smithi050.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-30T01:01:20.088 INFO:teuthology.orchestra.run.smithi050.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-30T01:01:20.088 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.a] 2023-09-30T01:01:20.088 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.c] 2023-09-30T01:01:20.088 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.b] 2023-09-30T01:01:20.088 INFO:tasks.ceph:Creating admin key on mon.a... 2023-09-30T01:01:20.088 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-09-30T01:01:20.167 INFO:tasks.ceph:Copying monmap to all nodes... 2023-09-30T01:01:20.167 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:20.168 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-09-30T01:01:20.173 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:20.173 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-09-30T01:01:20.180 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi037.front.sepia.ceph.com 2023-09-30T01:01:20.180 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:20.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-30T01:01:20.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T01:01:20.203 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:20.203 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:20.207 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi050.front.sepia.ceph.com 2023-09-30T01:01:20.208 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:20.208 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-30T01:01:20.208 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T01:01:20.235 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:20.236 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:20.241 INFO:tasks.ceph:Setting up mon nodes... 2023-09-30T01:01:20.241 INFO:tasks.ceph:Setting up mgr nodes... 2023-09-30T01:01:20.241 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-09-30T01:01:20.284 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-09-30T01:01:20.286 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-09-30T01:01:20.332 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-09-30T01:01:20.335 INFO:tasks.ceph:Setting up mds nodes... 2023-09-30T01:01:20.335 INFO:tasks.ceph_client:Setting up client nodes... 2023-09-30T01:01:20.335 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-30T01:01:20.370 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-09-30T01:01:20.384 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-09-30T01:01:20.421 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-09-30T01:01:20.434 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-09-30T01:01:20.434 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-30T01:01:20.434 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-09-30T01:01:20.451 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T01:01:20.451 INFO:tasks.ceph:role: osd.0 2023-09-30T01:01:20.451 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com 2023-09-30T01:01:20.452 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-30T01:01:21.002 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T01:01:21.003 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T01:01:21.003 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T01:01:21.003 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 2023-09-30T01:01:21.003 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T01:01:21.003 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2023-09-30T01:01:21.003 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T01:01:21.003 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T01:01:21.003 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T01:01:21.003 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T01:01:21.004 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2023-09-30T01:01:21.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-09-30T01:01:21.084 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-09-30T01:01:21.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:01:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T01:01:21.097 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-09-30T01:01:21.157 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T01:01:21.157 INFO:tasks.ceph:role: osd.1 2023-09-30T01:01:21.157 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com 2023-09-30T01:01:21.158 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-30T01:01:21.673 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T01:01:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T01:01:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T01:01:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 2023-09-30T01:01:21.673 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T01:01:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2023-09-30T01:01:21.674 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T01:01:21.674 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T01:01:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T01:01:21.674 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T01:01:21.675 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2023-09-30T01:01:21.675 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-09-30T01:01:21.721 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-09-30T01:01:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T01:01:21.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:01:21.780 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-09-30T01:01:21.790 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T01:01:21.790 INFO:tasks.ceph:role: osd.2 2023-09-30T01:01:21.790 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com 2023-09-30T01:01:21.790 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-30T01:01:22.302 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T01:01:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T01:01:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T01:01:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 2023-09-30T01:01:22.303 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T01:01:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2023-09-30T01:01:22.303 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T01:01:22.303 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T01:01:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T01:01:22.304 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T01:01:22.304 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2023-09-30T01:01:22.305 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-09-30T01:01:22.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-09-30T01:01:22.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:01:22.370 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T01:01:22.371 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-09-30T01:01:22.381 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T01:01:22.381 INFO:tasks.ceph:role: osd.3 2023-09-30T01:01:22.382 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com 2023-09-30T01:01:22.382 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-30T01:01:22.873 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T01:01:22.874 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T01:01:22.874 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T01:01:22.874 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 2023-09-30T01:01:22.874 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T01:01:22.874 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2023-09-30T01:01:22.875 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T01:01:22.875 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T01:01:22.875 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T01:01:22.875 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T01:01:22.876 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2023-09-30T01:01:22.877 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-09-30T01:01:22.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-09-30T01:01:22.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:01:22.984 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T01:01:22.984 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:23.071 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:23.066+0000 7fbabb54e3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-09-30T01:01:23.073 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:23.066+0000 7fbabb54e3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-09-30T01:01:23.073 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:23.066+0000 7fbabb54e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T01:01:23.073 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:23.066+0000 7fbabb54e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T01:01:23.073 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:23.066+0000 7fbabb54e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T01:01:23.079 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:23.074+0000 7fbabb54e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-09-30T01:01:23.894 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:23.890+0000 7fbabb54e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:24.682 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:24.678+0000 7fbabb54e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:25.462 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:25.458+0000 7fbabb54e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:26.314 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-09-30T01:01:26.330 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:26.414 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:26.410+0000 7f1e57ab03c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-09-30T01:01:26.415 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:26.410+0000 7f1e57ab03c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-09-30T01:01:26.415 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:26.410+0000 7f1e57ab03c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T01:01:26.415 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:26.410+0000 7f1e57ab03c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T01:01:26.415 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:26.410+0000 7f1e57ab03c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T01:01:26.415 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:26.410+0000 7f1e57ab03c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-09-30T01:01:26.991 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:26.986+0000 7f1e57ab03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:27.774 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:27.770+0000 7f1e57ab03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:28.555 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:28.550+0000 7f1e57ab03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:29.418 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-09-30T01:01:29.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:29.469 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:29.462+0000 7fc573d883c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-09-30T01:01:29.469 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:29.462+0000 7fc573d883c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-09-30T01:01:29.469 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:29.462+0000 7fc573d883c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T01:01:29.469 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:29.462+0000 7fc573d883c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T01:01:29.469 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:29.462+0000 7fc573d883c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T01:01:29.469 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:29.466+0000 7fc573d883c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-09-30T01:01:30.027 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:30.022+0000 7fc573d883c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:30.817 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:30.814+0000 7fc573d883c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:31.600 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:31.594+0000 7fc573d883c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:32.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-09-30T01:01:32.533 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:32.569 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:32.566+0000 7ff427c513c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-09-30T01:01:32.570 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:32.566+0000 7ff427c513c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-09-30T01:01:32.570 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:32.566+0000 7ff427c513c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T01:01:32.570 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:32.566+0000 7ff427c513c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T01:01:32.570 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:32.566+0000 7ff427c513c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T01:01:32.570 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:32.566+0000 7ff427c513c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-09-30T01:01:33.139 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:33.134+0000 7ff427c513c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:33.918 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:33.914+0000 7ff427c513c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:34.716 INFO:teuthology.orchestra.run.smithi037.stderr:2023-09-30T01:01:34.710+0000 7ff427c513c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:35.593 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-09-30T01:01:35.609 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-09-30T01:01:35.627 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-30T01:01:35.627 INFO:tasks.ceph:role: osd.4 2023-09-30T01:01:35.627 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com 2023-09-30T01:01:35.628 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-30T01:01:35.670 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T01:01:35.670 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T01:01:35.670 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T01:01:35.670 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 2023-09-30T01:01:35.670 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T01:01:35.670 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-09-30T01:01:35.670 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T01:01:35.670 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T01:01:35.671 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T01:01:35.671 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T01:01:35.671 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-09-30T01:01:35.672 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-09-30T01:01:35.736 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-09-30T01:01:35.748 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T01:01:35.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:01:35.749 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-09-30T01:01:35.760 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-30T01:01:35.760 INFO:tasks.ceph:role: osd.5 2023-09-30T01:01:35.760 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com 2023-09-30T01:01:35.761 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-30T01:01:35.847 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T01:01:35.847 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T01:01:35.848 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T01:01:35.848 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 2023-09-30T01:01:35.848 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T01:01:35.848 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-09-30T01:01:35.848 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T01:01:35.848 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T01:01:35.848 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T01:01:35.848 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T01:01:35.850 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-09-30T01:01:35.850 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-09-30T01:01:35.872 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-09-30T01:01:35.880 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T01:01:35.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:01:35.881 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-09-30T01:01:35.938 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-30T01:01:35.938 INFO:tasks.ceph:role: osd.6 2023-09-30T01:01:35.938 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com 2023-09-30T01:01:35.938 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-30T01:01:35.979 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T01:01:35.980 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T01:01:35.980 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T01:01:35.980 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 2023-09-30T01:01:35.980 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T01:01:35.980 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-09-30T01:01:35.980 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T01:01:35.980 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T01:01:35.980 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T01:01:35.980 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T01:01:35.981 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-09-30T01:01:35.982 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-09-30T01:01:35.997 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-09-30T01:01:36.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:01:36.057 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T01:01:36.057 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-09-30T01:01:36.070 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-30T01:01:36.070 INFO:tasks.ceph:role: osd.7 2023-09-30T01:01:36.070 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi050.front.sepia.ceph.com 2023-09-30T01:01:36.071 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-30T01:01:36.113 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T01:01:36.113 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T01:01:36.113 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T01:01:36.114 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 2023-09-30T01:01:36.114 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T01:01:36.114 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2023-09-30T01:01:36.114 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T01:01:36.114 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T01:01:36.114 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T01:01:36.114 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T01:01:36.115 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2023-09-30T01:01:36.115 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-09-30T01:01:36.135 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-09-30T01:01:36.191 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T01:01:36.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:01:36.193 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:36.259 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:36.254+0000 7fd73da7b3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-09-30T01:01:36.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:36.254+0000 7fd73da7b3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-09-30T01:01:36.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:36.254+0000 7fd73da7b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-30T01:01:36.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:36.254+0000 7fd73da7b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-30T01:01:36.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:36.254+0000 7fd73da7b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-30T01:01:36.260 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:36.258+0000 7fd73da7b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-09-30T01:01:36.817 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:36.814+0000 7fd73da7b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:37.604 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:37.602+0000 7fd73da7b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:38.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:38.394+0000 7fd73da7b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:39.257 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-09-30T01:01:39.273 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:39.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:39.310+0000 7fb9fdccf3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-09-30T01:01:39.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:39.310+0000 7fb9fdccf3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-09-30T01:01:39.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:39.310+0000 7fb9fdccf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-30T01:01:39.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:39.310+0000 7fb9fdccf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-30T01:01:39.314 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:39.310+0000 7fb9fdccf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-30T01:01:39.315 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:39.310+0000 7fb9fdccf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-09-30T01:01:39.873 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:39.870+0000 7fb9fdccf3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:40.651 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:40.646+0000 7fb9fdccf3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:41.440 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:41.438+0000 7fb9fdccf3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:42.296 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-09-30T01:01:42.359 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:42.396 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:42.394+0000 7ff3835f53c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-09-30T01:01:42.428 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:42.394+0000 7ff3835f53c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-09-30T01:01:42.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:42.394+0000 7ff3835f53c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-30T01:01:42.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:42.394+0000 7ff3835f53c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-30T01:01:42.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:42.394+0000 7ff3835f53c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-30T01:01:42.429 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:42.394+0000 7ff3835f53c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-09-30T01:01:42.958 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:42.954+0000 7ff3835f53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:43.736 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:43.734+0000 7ff3835f53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:44.524 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:44.522+0000 7ff3835f53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:45.372 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-09-30T01:01:45.387 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:45.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:45.470+0000 7fc7078f03c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-09-30T01:01:45.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:45.470+0000 7fc7078f03c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-09-30T01:01:45.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:45.470+0000 7fc7078f03c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-30T01:01:45.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:45.470+0000 7fc7078f03c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-30T01:01:45.473 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:45.470+0000 7fc7078f03c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-30T01:01:45.474 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:45.470+0000 7fc7078f03c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-09-30T01:01:46.028 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:46.022+0000 7fc7078f03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:46.816 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:46.814+0000 7fc7078f03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:47.593 INFO:teuthology.orchestra.run.smithi050.stderr:2023-09-30T01:01:47.590+0000 7fc7078f03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T01:01:48.459 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-09-30T01:01:48.476 INFO:tasks.ceph:Reading keys from all nodes... 2023-09-30T01:01:48.476 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:48.476 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-09-30T01:01:48.490 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:48.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-09-30T01:01:48.506 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:48.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-09-30T01:01:48.522 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:48.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-09-30T01:01:48.581 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:48.581 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-09-30T01:01:48.594 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:48.594 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-09-30T01:01:48.610 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:48.610 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-09-30T01:01:48.668 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:48.668 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-09-30T01:01:48.683 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:48.683 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-09-30T01:01:48.694 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:48.694 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-09-30T01:01:48.751 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:48.751 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-09-30T01:01:48.758 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:48.758 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-09-30T01:01:48.764 INFO:tasks.ceph:Adding keys to all mons... 2023-09-30T01:01:48.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-30T01:01:48.803 DEBUG:teuthology.orchestra.run.smithi050:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-30T01:01:48.815 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.y] 2023-09-30T01:01:48.815 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBgcxdl9wTmEBAAJNXMGG6Rz+pYk1ZHisZsfQ== 2023-09-30T01:01:48.815 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.0] 2023-09-30T01:01:48.815 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBjcxdl/Ps5BBAAYGi+C8Qlmv6zvSg4NpGFog== 2023-09-30T01:01:48.815 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.1] 2023-09-30T01:01:48.816 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBmcxdlJ1qmGBAA4QpeM0CFSo23WYLNxobjPQ== 2023-09-30T01:01:48.816 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.2] 2023-09-30T01:01:48.816 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBpcxdlKPLfGxAAguTmbr8APZit6+07FcAQZg== 2023-09-30T01:01:48.816 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.3] 2023-09-30T01:01:48.816 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBscxdlipTmIRAATQPW//pjLuHZXa4G50EV6Q== 2023-09-30T01:01:48.816 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.x] 2023-09-30T01:01:48.816 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBgcxdlNrrtExAAXBKjddE8LwnAAGe/z75Y1w== 2023-09-30T01:01:48.816 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.4] 2023-09-30T01:01:48.816 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBwcxdlq2SNDxAAMHdD+iBQmueWxwTHHcUgrA== 2023-09-30T01:01:48.816 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.5] 2023-09-30T01:01:48.816 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBzcxdl/aPOEhAA2RX87LiMcWA4+IU9HGcx0w== 2023-09-30T01:01:48.817 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.6] 2023-09-30T01:01:48.817 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQB2cxdlKV+2FxAAzoLl1CyvQHm1vnDjnkFLcQ== 2023-09-30T01:01:48.817 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.7] 2023-09-30T01:01:48.817 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQB5cxdluz1GHBAA5MCrTgnLfoJO+lo7w3CGuQ== 2023-09-30T01:01:48.817 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2023-09-30T01:01:48.817 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBgcxdl+kYMFhAAePP4H2BBmf2rlhJ5AGrjzg== 2023-09-30T01:01:48.817 INFO:teuthology.orchestra.run.smithi037.stdout:[client.1] 2023-09-30T01:01:48.817 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBgcxdlP0AvGRAAcXu0JvLTHTdc0utNTe3nqw== 2023-09-30T01:01:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr.y] 2023-09-30T01:01:48.822 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBgcxdl9wTmEBAAJNXMGG6Rz+pYk1ZHisZsfQ== 2023-09-30T01:01:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.0] 2023-09-30T01:01:48.822 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBjcxdl/Ps5BBAAYGi+C8Qlmv6zvSg4NpGFog== 2023-09-30T01:01:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.1] 2023-09-30T01:01:48.822 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBmcxdlJ1qmGBAA4QpeM0CFSo23WYLNxobjPQ== 2023-09-30T01:01:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.2] 2023-09-30T01:01:48.822 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBpcxdlKPLfGxAAguTmbr8APZit6+07FcAQZg== 2023-09-30T01:01:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.3] 2023-09-30T01:01:48.822 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBscxdlipTmIRAATQPW//pjLuHZXa4G50EV6Q== 2023-09-30T01:01:48.822 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr.x] 2023-09-30T01:01:48.822 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBgcxdlNrrtExAAXBKjddE8LwnAAGe/z75Y1w== 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.4] 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBwcxdlq2SNDxAAMHdD+iBQmueWxwTHHcUgrA== 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.5] 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBzcxdl/aPOEhAA2RX87LiMcWA4+IU9HGcx0w== 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.6] 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQB2cxdlKV+2FxAAzoLl1CyvQHm1vnDjnkFLcQ== 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.7] 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQB5cxdluz1GHBAA5MCrTgnLfoJO+lo7w3CGuQ== 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBgcxdl+kYMFhAAePP4H2BBmf2rlhJ5AGrjzg== 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout:[client.1] 2023-09-30T01:01:48.823 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBgcxdlP0AvGRAAcXu0JvLTHTdc0utNTe3nqw== 2023-09-30T01:01:48.825 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-30T01:01:48.827 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-30T01:01:48.870 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:48.915 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:48.963 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:48.965 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.076 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.079 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.124 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.163 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.211 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-30T01:01:49.214 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-30T01:01:49.259 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.262 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.304 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.347 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.397 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.442 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.444 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T01:01:49.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-30T01:01:49.531 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-30T01:01:49.578 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-30T01:01:49.581 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-30T01:01:49.627 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-09-30T01:01:49.627 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-09-30T01:01:49.638 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T01:01:49.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-09-30T01:01:49.757 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-09-30T01:01:49.768 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T01:01:49.869 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-09-30T01:01:49.882 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-09-30T01:01:49.899 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T01:01:50.002 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-09-30T01:01:50.018 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:50.020 DEBUG:teuthology.orchestra.run.smithi050:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-30T01:01:50.025 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-09-30T01:01:50.025 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-30T01:01:50.025 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-30T01:01:50.066 INFO:tasks.ceph.mon.a:Started 2023-09-30T01:01:50.067 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-30T01:01:50.067 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-30T01:01:50.069 INFO:tasks.ceph.mon.c:Started 2023-09-30T01:01:50.069 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-30T01:01:50.069 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-30T01:01:50.071 INFO:tasks.ceph.mon.b:Started 2023-09-30T01:01:50.072 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-09-30T01:01:50.072 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-09-30T01:01:50.072 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-09-30T01:01:50.075 INFO:tasks.ceph.mgr.y:Started 2023-09-30T01:01:50.075 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-30T01:01:50.075 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-30T01:01:50.077 INFO:tasks.ceph.mgr.x:Started 2023-09-30T01:01:50.077 INFO:tasks.ceph:Setting crush tunables to default 2023-09-30T01:01:50.077 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd crush tunables default 2023-09-30T01:01:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2023-09-30T01:01:55.550 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-09-30T01:01:55.550 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:55.550 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-09-30T01:01:55.562 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:55.562 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-09-30T01:01:55.574 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:55.575 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-09-30T01:01:55.586 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2023-09-30T01:01:55.586 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-09-30T01:01:55.600 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:55.600 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-09-30T01:01:55.616 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:55.616 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-09-30T01:01:55.617 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:55.613+0000 7f619efa6000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T01:01:55.618 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:55.613+0000 7f420b896000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T01:01:55.625 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:55.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-09-30T01:01:55.683 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2023-09-30T01:01:55.683 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-09-30T01:01:55.692 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new 2e24b7d6-a497-485e-b99b-0bfa85c43338 0 2023-09-30T01:01:55.889 INFO:teuthology.orchestra.run.smithi050.stdout:0 2023-09-30T01:01:55.902 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new a8f0a7a5-328d-4868-a30d-22e3b5b832ad 1 2023-09-30T01:01:56.099 INFO:teuthology.orchestra.run.smithi050.stdout:1 2023-09-30T01:01:56.111 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new 376a9ecf-3a2c-4bd5-add5-0d1e64bece9e 2 2023-09-30T01:01:56.222 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:56.217+0000 7f619efa6000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T01:01:56.283 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:56.277+0000 7f619efa6000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T01:01:56.316 INFO:teuthology.orchestra.run.smithi050.stdout:2 2023-09-30T01:01:56.329 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new 4b7b474d-a9a6-4d45-b263-da4ddeb0a5e7 3 2023-09-30T01:01:56.396 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:56.389+0000 7f619efa6000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T01:01:56.401 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:56.397+0000 7f420b896000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T01:01:56.459 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:56.453+0000 7f420b896000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T01:01:56.527 INFO:teuthology.orchestra.run.smithi050.stdout:3 2023-09-30T01:01:56.540 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new 949fb796-6e09-474e-b21e-8ba7a52c1d79 4 2023-09-30T01:01:56.576 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:56.569+0000 7f420b896000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T01:01:56.627 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:56.621+0000 7f619efa6000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T01:01:56.678 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:56.673+0000 7f619efa6000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T01:01:56.729 INFO:teuthology.orchestra.run.smithi050.stdout:4 2023-09-30T01:01:56.741 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new 869b70fb-be90-4d48-b9c6-a1f2d6e1682f 5 2023-09-30T01:01:56.826 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:56.821+0000 7f420b896000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T01:01:56.861 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:56.857+0000 7f619efa6000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T01:01:56.880 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:56.873+0000 7f420b896000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T01:01:56.925 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:56.921+0000 7f619efa6000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T01:01:56.938 INFO:teuthology.orchestra.run.smithi050.stdout:5 2023-09-30T01:01:56.953 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new bddff542-e807-42f4-91e1-c6b3db62d7a8 6 2023-09-30T01:01:56.990 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:56.985+0000 7f619efa6000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T01:01:57.144 INFO:teuthology.orchestra.run.smithi050.stdout:6 2023-09-30T01:01:57.158 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new a1448ff4-9ea4-4fd3-a940-ed51cbb9b3de 7 2023-09-30T01:01:57.189 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:57.181+0000 7f420b896000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T01:01:57.259 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:57.253+0000 7f420b896000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T01:01:57.328 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:57.321+0000 7f420b896000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T01:01:57.349 INFO:teuthology.orchestra.run.smithi050.stdout:7 2023-09-30T01:01:57.363 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-30T01:01:57.363 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-30T01:01:57.365 INFO:tasks.ceph.osd.0:Started 2023-09-30T01:01:57.365 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-30T01:01:57.365 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-30T01:01:57.367 INFO:tasks.ceph.osd.1:Started 2023-09-30T01:01:57.368 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-30T01:01:57.368 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-30T01:01:57.370 INFO:tasks.ceph.osd.2:Started 2023-09-30T01:01:57.370 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-30T01:01:57.370 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-30T01:01:57.372 INFO:tasks.ceph.osd.3:Started 2023-09-30T01:01:57.372 INFO:tasks.ceph.osd.4:Restarting daemon 2023-09-30T01:01:57.372 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-09-30T01:01:57.374 INFO:tasks.ceph.osd.4:Started 2023-09-30T01:01:57.374 INFO:tasks.ceph.osd.5:Restarting daemon 2023-09-30T01:01:57.374 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-09-30T01:01:57.379 INFO:tasks.ceph.osd.5:Started 2023-09-30T01:01:57.379 INFO:tasks.ceph.osd.6:Restarting daemon 2023-09-30T01:01:57.379 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-09-30T01:01:57.381 INFO:tasks.ceph.osd.6:Started 2023-09-30T01:01:57.381 INFO:tasks.ceph.osd.7:Restarting daemon 2023-09-30T01:01:57.381 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-09-30T01:01:57.382 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:57.377+0000 7f619efa6000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T01:01:57.383 INFO:tasks.ceph.osd.7:Started 2023-09-30T01:01:57.384 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T01:01:57.526 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:57.521+0000 7f619efa6000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T01:01:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:01:57.573 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"fsid":"43b20d8a-eabf-4c8a-988f-eaf5c9521071","created":"2023-09-30T01:01:55.405921+0000","modified":"2023-09-30T01:01:57.341399+0000","last_up_change":"0.000000","last_in_change":"2023-09-30T01:01:57.341399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2e24b7d6-a497-485e-b99b-0bfa85c43338","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a8f0a7a5-328d-4868-a30d-22e3b5b832ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"376a9ecf-3a2c-4bd5-add5-0d1e64bece9e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4b7b474d-a9a6-4d45-b263-da4ddeb0a5e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"949fb796-6e09-474e-b21e-8ba7a52c1d79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"869b70fb-be90-4d48-b9c6-a1f2d6e1682f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"bddff542-e807-42f4-91e1-c6b3db62d7a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a1448ff4-9ea4-4fd3-a940-ed51cbb9b3de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T01:01:57.585 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-09-30T01:01:57.585 INFO:tasks.ceph:Waiting for OSDs to come up 2023-09-30T01:01:57.743 INFO:tasks.ceph.osd.3.smithi037.stderr:2023-09-30T01:01:57.737+0000 7f932765f3c0 -1 Falling back to public interface 2023-09-30T01:01:57.744 INFO:tasks.ceph.osd.0.smithi037.stderr:2023-09-30T01:01:57.737+0000 7f8d545413c0 -1 Falling back to public interface 2023-09-30T01:01:57.747 INFO:tasks.ceph.osd.1.smithi037.stderr:2023-09-30T01:01:57.741+0000 7f5bf8e8d3c0 -1 Falling back to public interface 2023-09-30T01:01:57.758 INFO:tasks.ceph.osd.2.smithi037.stderr:2023-09-30T01:01:57.753+0000 7f5d89b563c0 -1 Falling back to public interface 2023-09-30T01:01:57.853 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:57.849+0000 7f619efa6000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T01:01:57.875 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:57.869+0000 7f420b896000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T01:01:57.986 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-09-30T01:01:57.981+0000 7f712c6c13c0 -1 Falling back to public interface 2023-09-30T01:01:57.994 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-09-30T01:01:57.989+0000 7f865f9073c0 -1 Falling back to public interface 2023-09-30T01:01:57.997 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:57.993+0000 7f420b896000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T01:01:58.008 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-09-30T01:01:58.005+0000 7f616e0453c0 -1 Falling back to public interface 2023-09-30T01:01:58.009 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-09-30T01:01:58.005+0000 7f50819a53c0 -1 Falling back to public interface 2023-09-30T01:01:58.028 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:58.025+0000 7f619efa6000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T01:01:58.096 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:58.093+0000 7f619efa6000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T01:01:58.149 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:58.145+0000 7f619efa6000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T01:01:58.207 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:58.201+0000 7f619efa6000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T01:01:58.320 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:58.313+0000 7f420b896000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T01:01:58.390 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-30T01:01:58.471 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:58.465+0000 7f619efa6000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T01:01:58.490 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:58.485+0000 7f420b896000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T01:01:58.530 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:58.525+0000 7f619efa6000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T01:01:58.556 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:58.549+0000 7f420b896000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T01:01:58.559 INFO:teuthology.misc.health.smithi037.stdout: 2023-09-30T01:01:58.559 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":10,"fsid":"43b20d8a-eabf-4c8a-988f-eaf5c9521071","created":"2023-09-30T01:01:55.405921+0000","modified":"2023-09-30T01:01:57.341399+0000","last_up_change":"0.000000","last_in_change":"2023-09-30T01:01:57.341399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2e24b7d6-a497-485e-b99b-0bfa85c43338","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a8f0a7a5-328d-4868-a30d-22e3b5b832ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"376a9ecf-3a2c-4bd5-add5-0d1e64bece9e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4b7b474d-a9a6-4d45-b263-da4ddeb0a5e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"949fb796-6e09-474e-b21e-8ba7a52c1d79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"869b70fb-be90-4d48-b9c6-a1f2d6e1682f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"bddff542-e807-42f4-91e1-c6b3db62d7a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a1448ff4-9ea4-4fd3-a940-ed51cbb9b3de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T01:01:58.570 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-09-30T01:01:58.598 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:58.593+0000 7f619efa6000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T01:01:58.607 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:58.601+0000 7f420b896000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T01:01:58.665 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:58.657+0000 7f420b896000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T01:01:58.764 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:58.761+0000 7f619efa6000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T01:01:58.832 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:58.829+0000 7f619efa6000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T01:01:58.945 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:58.941+0000 7f420b896000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T01:01:59.004 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:58.997+0000 7f420b896000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T01:01:59.072 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:59.065+0000 7f420b896000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T01:01:59.238 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:59.233+0000 7f420b896000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T01:01:59.306 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:01:59.301+0000 7f420b896000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T01:01:59.559 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:59.553+0000 7f619efa6000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T01:01:59.615 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:59.609+0000 7f617c887700 -1 log_channel(cluster) log [ERR] : Unhandled exception from module 'pg_autoscaler' while running on mgr.x: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-30T01:01:59.616 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:59.609+0000 7f617c887700 -1 pg_autoscaler.serve: 2023-09-30T01:01:59.618 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:01:59.609+0000 7f617c887700 -1 Traceback (most recent call last): 2023-09-30T01:01:59.618 INFO:tasks.ceph.mgr.x.smithi050.stderr: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 325, in serve 2023-09-30T01:01:59.618 INFO:tasks.ceph.mgr.x.smithi050.stderr: self._maybe_adjust() 2023-09-30T01:01:59.618 INFO:tasks.ceph.mgr.x.smithi050.stderr: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 682, in _maybe_adjust 2023-09-30T01:01:59.618 INFO:tasks.ceph.mgr.x.smithi050.stderr: if self.noautoscale: 2023-09-30T01:01:59.619 INFO:tasks.ceph.mgr.x.smithi050.stderr:AttributeError: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-30T01:01:59.619 INFO:tasks.ceph.mgr.x.smithi050.stderr: 2023-09-30T01:02:00.037 INFO:tasks.ceph.mgr.y.smithi037.stderr:2023-09-30T01:02:00.029+0000 7f420b896000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T01:02:00.602 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:02:00.597+0000 7f618809e700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-30T01:02:00.602 INFO:tasks.ceph.mgr.x.smithi050.stderr:2023-09-30T01:02:00.597+0000 7f618809e700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-30T01:02:01.327 INFO:tasks.ceph.osd.3.smithi037.stderr:2023-09-30T01:02:01.321+0000 7f932765f3c0 -1 osd.3 0 log_to_monitors true 2023-09-30T01:02:01.347 INFO:tasks.ceph.osd.0.smithi037.stderr:2023-09-30T01:02:01.341+0000 7f8d545413c0 -1 osd.0 0 log_to_monitors true 2023-09-30T01:02:01.385 INFO:tasks.ceph.osd.2.smithi037.stderr:2023-09-30T01:02:01.377+0000 7f5d89b563c0 -1 osd.2 0 log_to_monitors true 2023-09-30T01:02:01.525 INFO:tasks.ceph.osd.1.smithi037.stderr:2023-09-30T01:02:01.517+0000 7f5bf8e8d3c0 -1 osd.1 0 log_to_monitors true 2023-09-30T01:02:01.565 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-09-30T01:02:01.561+0000 7f712c6c13c0 -1 osd.5 0 log_to_monitors true 2023-09-30T01:02:01.798 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-09-30T01:02:01.793+0000 7f616e0453c0 -1 osd.7 0 log_to_monitors true 2023-09-30T01:02:01.827 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-09-30T01:02:01.821+0000 7f50819a53c0 -1 osd.6 0 log_to_monitors true 2023-09-30T01:02:01.832 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-09-30T01:02:01.829+0000 7f865f9073c0 -1 osd.4 0 log_to_monitors true 2023-09-30T01:02:02.608 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-09-30T01:02:02.605+0000 7f7129686700 -1 osd.5 0 waiting for initial osdmap 2023-09-30T01:02:02.610 INFO:tasks.ceph.osd.1.smithi037.stderr:2023-09-30T01:02:02.605+0000 7f5bf5e52700 -1 osd.1 0 waiting for initial osdmap 2023-09-30T01:02:02.612 INFO:tasks.ceph.osd.0.smithi037.stderr:2023-09-30T01:02:02.605+0000 7f8d4fce1700 -1 osd.0 0 waiting for initial osdmap 2023-09-30T01:02:02.616 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-09-30T01:02:02.613+0000 7f711e62d700 -1 osd.5 0 handle_osd_map injecting map crc failure 2023-09-30T01:02:02.618 INFO:tasks.ceph.osd.1.smithi037.stderr:2023-09-30T01:02:02.609+0000 7f5beadf9700 -1 osd.1 0 handle_osd_map injecting map crc failure 2023-09-30T01:02:02.619 INFO:tasks.ceph.osd.0.smithi037.stderr:2023-09-30T01:02:02.609+0000 7f8d45cac700 -1 osd.0 0 handle_osd_map injecting map crc failure 2023-09-30T01:02:02.620 INFO:tasks.ceph.osd.2.smithi037.stderr:2023-09-30T01:02:02.613+0000 7f5d852f6700 -1 osd.2 0 waiting for initial osdmap 2023-09-30T01:02:02.620 INFO:tasks.ceph.osd.3.smithi037.stderr:2023-09-30T01:02:02.613+0000 7f9322dff700 -1 osd.3 0 waiting for initial osdmap 2023-09-30T01:02:02.621 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-09-30T01:02:02.617+0000 7f712340a700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T01:02:02.622 INFO:tasks.ceph.osd.3.smithi037.stderr:2023-09-30T01:02:02.617+0000 7f93195cb700 -1 osd.3 0 handle_osd_map injecting map crc failure 2023-09-30T01:02:02.624 INFO:tasks.ceph.osd.1.smithi037.stderr:2023-09-30T01:02:02.617+0000 7f5befbd6700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T01:02:02.625 INFO:tasks.ceph.osd.2.smithi037.stderr:2023-09-30T01:02:02.617+0000 7f5d7bac2700 -1 osd.2 0 handle_osd_map injecting map crc failure 2023-09-30T01:02:02.626 INFO:tasks.ceph.osd.0.smithi037.stderr:2023-09-30T01:02:02.617+0000 7f8d4aa89700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T01:02:02.627 INFO:tasks.ceph.osd.3.smithi037.stderr:2023-09-30T01:02:02.621+0000 7f931e3a8700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T01:02:02.627 INFO:tasks.ceph.osd.2.smithi037.stderr:2023-09-30T01:02:02.621+0000 7f5d8089f700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T01:02:03.613 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-09-30T01:02:03.609+0000 7f61697e5700 -1 osd.7 0 waiting for initial osdmap 2023-09-30T01:02:03.615 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-09-30T01:02:03.609+0000 7f507d145700 -1 osd.6 0 waiting for initial osdmap 2023-09-30T01:02:03.615 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-09-30T01:02:03.609+0000 7f865b0a7700 -1 osd.4 0 waiting for initial osdmap 2023-09-30T01:02:03.616 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-09-30T01:02:03.613+0000 7f5074112700 -1 osd.6 0 handle_osd_map injecting map crc failure 2023-09-30T01:02:03.619 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-09-30T01:02:03.613+0000 7f8652074700 -1 osd.4 0 handle_osd_map injecting map crc failure 2023-09-30T01:02:03.619 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-09-30T01:02:03.613+0000 7f61607b2700 -1 osd.7 0 handle_osd_map injecting map crc failure 2023-09-30T01:02:03.622 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-09-30T01:02:03.617+0000 7f616558f700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T01:02:03.623 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-09-30T01:02:03.617+0000 7f8656e51700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T01:02:03.623 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-09-30T01:02:03.617+0000 7f5078eef700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T01:02:03.656 INFO:tasks.ceph.mon.a.smithi037.stderr:2023-09-30T01:02:03.649+0000 7fd4a2a5d700 -1 log_channel(cluster) log [ERR] : Health check failed: Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale' (MGR_MODULE_ERROR) 2023-09-30T01:02:05.377 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-30T01:02:05.642 INFO:tasks.ceph.osd.3.smithi037.stderr:2023-09-30T01:02:05.637+0000 7f93195cb700 -1 osd.3 13 handle_osd_map injecting map crc failure 2023-09-30T01:02:05.664 INFO:teuthology.misc.health.smithi037.stdout: 2023-09-30T01:02:05.665 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":15,"fsid":"43b20d8a-eabf-4c8a-988f-eaf5c9521071","created":"2023-09-30T01:01:55.405921+0000","modified":"2023-09-30T01:02:05.624594+0000","last_up_change":"2023-09-30T01:02:04.607879+0000","last_in_change":"2023-09-30T01:01:57.341399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T01:02:03.654754+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"2e24b7d6-a497-485e-b99b-0bfa85c43338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6802","nonce":31154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6804","nonce":31154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6807","nonce":31154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6805","nonce":31154}]},"public_addr":"172.21.15.37:6802/31154","cluster_addr":"172.21.15.37:6804/31154","heartbeat_back_addr":"172.21.15.37:6807/31154","heartbeat_front_addr":"172.21.15.37:6805/31154","state":["exists","up"]},{"osd":1,"uuid":"a8f0a7a5-328d-4868-a30d-22e3b5b832ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6808","nonce":31153}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6809","nonce":31153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6811","nonce":31153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6810","nonce":31153}]},"public_addr":"172.21.15.37:6808/31153","cluster_addr":"172.21.15.37:6809/31153","heartbeat_back_addr":"172.21.15.37:6811/31153","heartbeat_front_addr":"172.21.15.37:6810/31153","state":["exists","up"]},{"osd":2,"uuid":"376a9ecf-3a2c-4bd5-add5-0d1e64bece9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6812","nonce":31155}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6813","nonce":31155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6815","nonce":31155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6814","nonce":31155}]},"public_addr":"172.21.15.37:6812/31155","cluster_addr":"172.21.15.37:6813/31155","heartbeat_back_addr":"172.21.15.37:6815/31155","heartbeat_front_addr":"172.21.15.37:6814/31155","state":["exists","up"]},{"osd":3,"uuid":"4b7b474d-a9a6-4d45-b263-da4ddeb0a5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6800","nonce":31152}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6801","nonce":31152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6806","nonce":31152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6803","nonce":31152}]},"public_addr":"172.21.15.37:6800/31152","cluster_addr":"172.21.15.37:6801/31152","heartbeat_back_addr":"172.21.15.37:6806/31152","heartbeat_front_addr":"172.21.15.37:6803/31152","state":["exists","up"]},{"osd":4,"uuid":"949fb796-6e09-474e-b21e-8ba7a52c1d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6804","nonce":31190}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6805","nonce":31190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6807","nonce":31190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6806","nonce":31190}]},"public_addr":"172.21.15.50:6804/31190","cluster_addr":"172.21.15.50:6805/31190","heartbeat_back_addr":"172.21.15.50:6807/31190","heartbeat_front_addr":"172.21.15.50:6806/31190","state":["exists","up"]},{"osd":5,"uuid":"869b70fb-be90-4d48-b9c6-a1f2d6e1682f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6800","nonce":31191}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6801","nonce":31191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6803","nonce":31191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6802","nonce":31191}]},"public_addr":"172.21.15.50:6800/31191","cluster_addr":"172.21.15.50:6801/31191","heartbeat_back_addr":"172.21.15.50:6803/31191","heartbeat_front_addr":"172.21.15.50:6802/31191","state":["exists","up"]},{"osd":6,"uuid":"bddff542-e807-42f4-91e1-c6b3db62d7a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6810","nonce":31192}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6812","nonce":31192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6815","nonce":31192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6813","nonce":31192}]},"public_addr":"172.21.15.50:6810/31192","cluster_addr":"172.21.15.50:6812/31192","heartbeat_back_addr":"172.21.15.50:6815/31192","heartbeat_front_addr":"172.21.15.50:6813/31192","state":["exists","up"]},{"osd":7,"uuid":"a1448ff4-9ea4-4fd3-a940-ed51cbb9b3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6808","nonce":31215}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6809","nonce":31215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6814","nonce":31215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6811","nonce":31215}]},"public_addr":"172.21.15.50:6808/31215","cluster_addr":"172.21.15.50:6809/31215","heartbeat_back_addr":"172.21.15.50:6814/31215","heartbeat_front_addr":"172.21.15.50:6811/31215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.318151+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.482430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.376699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.345608+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.820724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.540250+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.860697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.816773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T01:02:05.681 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-09-30T01:02:05.681 INFO:tasks.ceph:Creating RBD pool 2023-09-30T01:02:05.681 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd pool create rbd 8 2023-09-30T01:02:06.662 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'rbd' created 2023-09-30T01:02:06.674 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-09-30T01:02:06.810 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-09-30T01:02:06.805+0000 7f5074112700 -1 osd.6 14 handle_osd_map injecting map crc failure 2023-09-30T01:02:07.131 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-09-30T01:02:07.129+0000 7f711c629700 -1 osd.5 13 handle_osd_map injecting map crc failure 2023-09-30T01:02:07.670 INFO:tasks.ceph.osd.2.smithi037.stderr:2023-09-30T01:02:07.665+0000 7f5d7bac2700 -1 osd.2 16 handle_osd_map injecting map crc failure 2023-09-30T01:02:07.676 INFO:teuthology.orchestra.run.smithi037.stderr:enabled application 'rbd' on pool 'rbd' 2023-09-30T01:02:07.687 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-09-30T01:02:07.687 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T01:02:07.687 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-30T01:02:07.687 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-30T01:02:07.688 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-09-30T01:02:08.014 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:08.028 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":4,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6816","nonce":30893}]},"active_addr":"172.21.15.50:6816/30893","active_change":"2023-09-30T01:01:59.576565+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4109,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":3847454290}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":578904850}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":1499858444}]}]} 2023-09-30T01:02:08.034 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-30T01:02:08.035 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T01:02:08.035 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T01:02:08.325 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:08.325 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"fsid":"43b20d8a-eabf-4c8a-988f-eaf5c9521071","created":"2023-09-30T01:01:55.405921+0000","modified":"2023-09-30T01:02:07.650266+0000","last_up_change":"2023-09-30T01:02:04.607879+0000","last_in_change":"2023-09-30T01:01:57.341399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T01:02:03.654754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T01:02:06.017647+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2e24b7d6-a497-485e-b99b-0bfa85c43338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6802","nonce":31154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6804","nonce":31154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6807","nonce":31154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6805","nonce":31154}]},"public_addr":"172.21.15.37:6802/31154","cluster_addr":"172.21.15.37:6804/31154","heartbeat_back_addr":"172.21.15.37:6807/31154","heartbeat_front_addr":"172.21.15.37:6805/31154","state":["exists","up"]},{"osd":1,"uuid":"a8f0a7a5-328d-4868-a30d-22e3b5b832ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6808","nonce":31153}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6809","nonce":31153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6811","nonce":31153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6810","nonce":31153}]},"public_addr":"172.21.15.37:6808/31153","cluster_addr":"172.21.15.37:6809/31153","heartbeat_back_addr":"172.21.15.37:6811/31153","heartbeat_front_addr":"172.21.15.37:6810/31153","state":["exists","up"]},{"osd":2,"uuid":"376a9ecf-3a2c-4bd5-add5-0d1e64bece9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6812","nonce":31155}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6813","nonce":31155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6815","nonce":31155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6814","nonce":31155}]},"public_addr":"172.21.15.37:6812/31155","cluster_addr":"172.21.15.37:6813/31155","heartbeat_back_addr":"172.21.15.37:6815/31155","heartbeat_front_addr":"172.21.15.37:6814/31155","state":["exists","up"]},{"osd":3,"uuid":"4b7b474d-a9a6-4d45-b263-da4ddeb0a5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6800","nonce":31152}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6801","nonce":31152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6806","nonce":31152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6803","nonce":31152}]},"public_addr":"172.21.15.37:6800/31152","cluster_addr":"172.21.15.37:6801/31152","heartbeat_back_addr":"172.21.15.37:6806/31152","heartbeat_front_addr":"172.21.15.37:6803/31152","state":["exists","up"]},{"osd":4,"uuid":"949fb796-6e09-474e-b21e-8ba7a52c1d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6804","nonce":31190}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6805","nonce":31190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6807","nonce":31190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6806","nonce":31190}]},"public_addr":"172.21.15.50:6804/31190","cluster_addr":"172.21.15.50:6805/31190","heartbeat_back_addr":"172.21.15.50:6807/31190","heartbeat_front_addr":"172.21.15.50:6806/31190","state":["exists","up"]},{"osd":5,"uuid":"869b70fb-be90-4d48-b9c6-a1f2d6e1682f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6800","nonce":31191}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6801","nonce":31191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6803","nonce":31191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6802","nonce":31191}]},"public_addr":"172.21.15.50:6800/31191","cluster_addr":"172.21.15.50:6801/31191","heartbeat_back_addr":"172.21.15.50:6803/31191","heartbeat_front_addr":"172.21.15.50:6802/31191","state":["exists","up"]},{"osd":6,"uuid":"bddff542-e807-42f4-91e1-c6b3db62d7a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6810","nonce":31192}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6812","nonce":31192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6815","nonce":31192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6813","nonce":31192}]},"public_addr":"172.21.15.50:6810/31192","cluster_addr":"172.21.15.50:6812/31192","heartbeat_back_addr":"172.21.15.50:6815/31192","heartbeat_front_addr":"172.21.15.50:6813/31192","state":["exists","up"]},{"osd":7,"uuid":"a1448ff4-9ea4-4fd3-a940-ed51cbb9b3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6808","nonce":31215}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6809","nonce":31215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6814","nonce":31215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6811","nonce":31215}]},"public_addr":"172.21.15.50:6808/31215","cluster_addr":"172.21.15.50:6809/31215","heartbeat_back_addr":"172.21.15.50:6814/31215","heartbeat_front_addr":"172.21.15.50:6811/31215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.318151+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.482430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.376699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.345608+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.820724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.540250+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.860697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.816773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T01:02:08.340 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T01:02:08.340 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T01:02:08.618 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:08.619 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"fsid":"43b20d8a-eabf-4c8a-988f-eaf5c9521071","created":"2023-09-30T01:01:55.405921+0000","modified":"2023-09-30T01:02:07.650266+0000","last_up_change":"2023-09-30T01:02:04.607879+0000","last_in_change":"2023-09-30T01:01:57.341399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T01:02:03.654754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T01:02:06.017647+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2e24b7d6-a497-485e-b99b-0bfa85c43338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6802","nonce":31154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6804","nonce":31154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6807","nonce":31154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6805","nonce":31154}]},"public_addr":"172.21.15.37:6802/31154","cluster_addr":"172.21.15.37:6804/31154","heartbeat_back_addr":"172.21.15.37:6807/31154","heartbeat_front_addr":"172.21.15.37:6805/31154","state":["exists","up"]},{"osd":1,"uuid":"a8f0a7a5-328d-4868-a30d-22e3b5b832ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6808","nonce":31153}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6809","nonce":31153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6811","nonce":31153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6810","nonce":31153}]},"public_addr":"172.21.15.37:6808/31153","cluster_addr":"172.21.15.37:6809/31153","heartbeat_back_addr":"172.21.15.37:6811/31153","heartbeat_front_addr":"172.21.15.37:6810/31153","state":["exists","up"]},{"osd":2,"uuid":"376a9ecf-3a2c-4bd5-add5-0d1e64bece9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6812","nonce":31155}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6813","nonce":31155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6815","nonce":31155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6814","nonce":31155}]},"public_addr":"172.21.15.37:6812/31155","cluster_addr":"172.21.15.37:6813/31155","heartbeat_back_addr":"172.21.15.37:6815/31155","heartbeat_front_addr":"172.21.15.37:6814/31155","state":["exists","up"]},{"osd":3,"uuid":"4b7b474d-a9a6-4d45-b263-da4ddeb0a5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6800","nonce":31152}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6801","nonce":31152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6806","nonce":31152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6803","nonce":31152}]},"public_addr":"172.21.15.37:6800/31152","cluster_addr":"172.21.15.37:6801/31152","heartbeat_back_addr":"172.21.15.37:6806/31152","heartbeat_front_addr":"172.21.15.37:6803/31152","state":["exists","up"]},{"osd":4,"uuid":"949fb796-6e09-474e-b21e-8ba7a52c1d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6804","nonce":31190}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6805","nonce":31190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6807","nonce":31190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6806","nonce":31190}]},"public_addr":"172.21.15.50:6804/31190","cluster_addr":"172.21.15.50:6805/31190","heartbeat_back_addr":"172.21.15.50:6807/31190","heartbeat_front_addr":"172.21.15.50:6806/31190","state":["exists","up"]},{"osd":5,"uuid":"869b70fb-be90-4d48-b9c6-a1f2d6e1682f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6800","nonce":31191}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6801","nonce":31191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6803","nonce":31191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6802","nonce":31191}]},"public_addr":"172.21.15.50:6800/31191","cluster_addr":"172.21.15.50:6801/31191","heartbeat_back_addr":"172.21.15.50:6803/31191","heartbeat_front_addr":"172.21.15.50:6802/31191","state":["exists","up"]},{"osd":6,"uuid":"bddff542-e807-42f4-91e1-c6b3db62d7a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6810","nonce":31192}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6812","nonce":31192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6815","nonce":31192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6813","nonce":31192}]},"public_addr":"172.21.15.50:6810/31192","cluster_addr":"172.21.15.50:6812/31192","heartbeat_back_addr":"172.21.15.50:6815/31192","heartbeat_front_addr":"172.21.15.50:6813/31192","state":["exists","up"]},{"osd":7,"uuid":"a1448ff4-9ea4-4fd3-a940-ed51cbb9b3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6808","nonce":31215}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6809","nonce":31215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6814","nonce":31215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6811","nonce":31215}]},"public_addr":"172.21.15.50:6808/31215","cluster_addr":"172.21.15.50:6809/31215","heartbeat_back_addr":"172.21.15.50:6814/31215","heartbeat_front_addr":"172.21.15.50:6811/31215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.318151+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.482430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.376699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.345608+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.820724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.540250+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.860697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:02:02.816773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T01:02:08.634 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-30T01:02:08.634 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-30T01:02:08.635 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-30T01:02:08.635 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-30T01:02:08.636 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-09-30T01:02:08.636 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-09-30T01:02:08.637 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-09-30T01:02:08.637 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-09-30T01:02:08.666 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-09-30T01:02:08.661+0000 7f8652074700 -1 osd.4 16 handle_osd_map injecting map crc failure 2023-09-30T01:02:08.850 INFO:teuthology.orchestra.run.smithi037.stdout:60129542147 2023-09-30T01:02:08.850 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-30T01:02:08.853 INFO:teuthology.orchestra.run.smithi037.stdout:55834574851 2023-09-30T01:02:08.853 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T01:02:08.859 INFO:teuthology.orchestra.run.smithi037.stdout:55834574851 2023-09-30T01:02:08.859 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-30T01:02:08.860 INFO:teuthology.orchestra.run.smithi037.stdout:55834574851 2023-09-30T01:02:08.861 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T01:02:08.862 INFO:teuthology.orchestra.run.smithi037.stdout:60129542148 2023-09-30T01:02:08.862 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-30T01:02:08.864 INFO:teuthology.orchestra.run.smithi037.stdout:55834574851 2023-09-30T01:02:08.864 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T01:02:08.870 INFO:teuthology.orchestra.run.smithi037.stdout:60129542147 2023-09-30T01:02:08.870 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-30T01:02:08.872 INFO:teuthology.orchestra.run.smithi037.stdout:55834574852 2023-09-30T01:02:08.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T01:02:09.316 INFO:teuthology.orchestra.run.smithi037.stdout:60129542146 2023-09-30T01:02:09.324 INFO:teuthology.orchestra.run.smithi037.stdout:55834574849 2023-09-30T01:02:09.341 INFO:teuthology.orchestra.run.smithi037.stdout:55834574849 2023-09-30T01:02:09.342 INFO:teuthology.orchestra.run.smithi037.stdout:55834574849 2023-09-30T01:02:09.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.6 2023-09-30T01:02:09.343 INFO:teuthology.orchestra.run.smithi037.stdout:55834574849 2023-09-30T01:02:09.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.0 2023-09-30T01:02:09.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.5 2023-09-30T01:02:09.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.2 2023-09-30T01:02:09.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.3 2023-09-30T01:02:09.357 INFO:teuthology.orchestra.run.smithi037.stdout:60129542147 2023-09-30T01:02:09.367 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2023-09-30T01:02:09.390 INFO:teuthology.orchestra.run.smithi037.stdout:60129542146 2023-09-30T01:02:09.394 INFO:teuthology.orchestra.run.smithi037.stdout:55834574850 2023-09-30T01:02:09.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.1 2023-09-30T01:02:09.409 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.7 2023-09-30T01:02:10.002 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-09-30T01:02:10.001+0000 7f507210e700 -1 osd.6 17 handle_osd_map injecting map crc failure 2023-09-30T01:02:10.343 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-30T01:02:10.344 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T01:02:10.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-30T01:02:10.357 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T01:02:10.357 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T01:02:10.369 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-30T01:02:10.407 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T01:02:10.410 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-30T01:02:10.800 INFO:teuthology.orchestra.run.smithi037.stdout:60129542147 2023-09-30T01:02:10.803 INFO:teuthology.orchestra.run.smithi037.stdout:55834574851 2023-09-30T01:02:10.827 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2023-09-30T01:02:10.827 DEBUG:teuthology.parallel:result is None 2023-09-30T01:02:10.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.6 2023-09-30T01:02:10.828 DEBUG:teuthology.parallel:result is None 2023-09-30T01:02:10.835 INFO:teuthology.orchestra.run.smithi037.stdout:60129542148 2023-09-30T01:02:10.839 INFO:teuthology.orchestra.run.smithi037.stdout:55834574851 2023-09-30T01:02:10.840 INFO:teuthology.orchestra.run.smithi037.stdout:55834574851 2023-09-30T01:02:10.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2023-09-30T01:02:10.854 DEBUG:teuthology.parallel:result is None 2023-09-30T01:02:10.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.4 2023-09-30T01:02:10.854 DEBUG:teuthology.parallel:result is None 2023-09-30T01:02:10.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2023-09-30T01:02:10.859 DEBUG:teuthology.parallel:result is None 2023-09-30T01:02:10.862 INFO:teuthology.orchestra.run.smithi037.stdout:55834574851 2023-09-30T01:02:10.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2023-09-30T01:02:10.874 DEBUG:teuthology.parallel:result is None 2023-09-30T01:02:10.887 INFO:teuthology.orchestra.run.smithi037.stdout:60129542147 2023-09-30T01:02:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:55834574852 2023-09-30T01:02:10.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.7 2023-09-30T01:02:10.899 DEBUG:teuthology.parallel:result is None 2023-09-30T01:02:10.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2023-09-30T01:02:10.907 DEBUG:teuthology.parallel:result is None 2023-09-30T01:02:10.907 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T01:02:10.908 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:02:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:11.182 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2023-09-30T01:02:11.193 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-09-30T01:02:09.590605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":197132,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":41,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":40780,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":39744,"kb_avail":754933940,"statfs":{"total":773094113280,"available":773052354560,"internally_reserved":0,"allocated":995328,"data_stored":588917,"data_compressed":720,"data_compressed_allocated":16384,"data_compressed_original":262144,"omap_allocated":0,"internal_metadata":40697856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.769880+0000","last_change":"2023-09-30T01:02:07.769880+0000","last_active":"2023-09-30T01:02:07.769880+0000","last_peered":"2023-09-30T01:02:07.769880+0000","last_clean":"2023-09-30T01:02:07.769880+0000","last_became_active":"2023-09-30T01:02:07.769178+0000","last_became_peered":"2023-09-30T01:02:07.769178+0000","last_unstale":"2023-09-30T01:02:07.769880+0000","last_undegraded":"2023-09-30T01:02:07.769880+0000","last_fullsized":"2023-09-30T01:02:07.769880+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:26.042079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.873156+0000","last_change":"2023-09-30T01:02:07.873156+0000","last_active":"2023-09-30T01:02:07.873156+0000","last_peered":"2023-09-30T01:02:07.873156+0000","last_clean":"2023-09-30T01:02:07.873156+0000","last_became_active":"2023-09-30T01:02:07.872154+0000","last_became_peered":"2023-09-30T01:02:07.872154+0000","last_unstale":"2023-09-30T01:02:07.873156+0000","last_undegraded":"2023-09-30T01:02:07.873156+0000","last_fullsized":"2023-09-30T01:02:07.873156+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:13.614779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.875706+0000","last_change":"2023-09-30T01:02:07.874353+0000","last_active":"2023-09-30T01:02:07.875706+0000","last_peered":"2023-09-30T01:02:07.875706+0000","last_clean":"2023-09-30T01:02:07.875706+0000","last_became_active":"2023-09-30T01:02:07.873496+0000","last_became_peered":"2023-09-30T01:02:07.873496+0000","last_unstale":"2023-09-30T01:02:07.875706+0000","last_undegraded":"2023-09-30T01:02:07.875706+0000","last_fullsized":"2023-09-30T01:02:07.875706+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:34.827586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.973236+0000","last_change":"2023-09-30T01:02:07.973236+0000","last_active":"2023-09-30T01:02:07.973236+0000","last_peered":"2023-09-30T01:02:07.973236+0000","last_clean":"2023-09-30T01:02:07.973236+0000","last_became_active":"2023-09-30T01:02:07.872317+0000","last_became_peered":"2023-09-30T01:02:07.872317+0000","last_unstale":"2023-09-30T01:02:07.973236+0000","last_undegraded":"2023-09-30T01:02:07.973236+0000","last_fullsized":"2023-09-30T01:02:07.973236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:14.385712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.878185+0000","last_change":"2023-09-30T01:02:07.878185+0000","last_active":"2023-09-30T01:02:07.878185+0000","last_peered":"2023-09-30T01:02:07.878185+0000","last_clean":"2023-09-30T01:02:07.878185+0000","last_became_active":"2023-09-30T01:02:07.677592+0000","last_became_peered":"2023-09-30T01:02:07.677592+0000","last_unstale":"2023-09-30T01:02:07.878185+0000","last_undegraded":"2023-09-30T01:02:07.878185+0000","last_fullsized":"2023-09-30T01:02:07.878185+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:14.385712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.775982+0000","last_change":"2023-09-30T01:02:07.775982+0000","last_active":"2023-09-30T01:02:07.775982+0000","last_peered":"2023-09-30T01:02:07.775982+0000","last_clean":"2023-09-30T01:02:07.775982+0000","last_became_active":"2023-09-30T01:02:07.775147+0000","last_became_peered":"2023-09-30T01:02:07.775147+0000","last_unstale":"2023-09-30T01:02:07.775982+0000","last_undegraded":"2023-09-30T01:02:07.775982+0000","last_fullsized":"2023-09-30T01:02:07.775982+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:27.018152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.873601+0000","last_change":"2023-09-30T01:02:07.773140+0000","last_active":"2023-09-30T01:02:07.873601+0000","last_peered":"2023-09-30T01:02:07.873601+0000","last_clean":"2023-09-30T01:02:07.873601+0000","last_became_active":"2023-09-30T01:02:07.772379+0000","last_became_peered":"2023-09-30T01:02:07.772379+0000","last_unstale":"2023-09-30T01:02:07.873601+0000","last_undegraded":"2023-09-30T01:02:07.873601+0000","last_fullsized":"2023-09-30T01:02:07.873601+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:09.142025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.774994+0000","last_change":"2023-09-30T01:02:07.774994+0000","last_active":"2023-09-30T01:02:07.774994+0000","last_peered":"2023-09-30T01:02:07.774994+0000","last_clean":"2023-09-30T01:02:07.774994+0000","last_became_active":"2023-09-30T01:02:07.773993+0000","last_became_peered":"2023-09-30T01:02:07.773993+0000","last_unstale":"2023-09-30T01:02:07.774994+0000","last_undegraded":"2023-09-30T01:02:07.774994+0000","last_fullsized":"2023-09-30T01:02:07.774994+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:36.638320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'20","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:08.699728+0000","last_change":"2023-09-30T01:02:06.437512+0000","last_active":"2023-09-30T01:02:08.699728+0000","last_peered":"2023-09-30T01:02:08.699728+0000","last_clean":"2023-09-30T01:02:08.699728+0000","last_became_active":"2023-09-30T01:02:06.336950+0000","last_became_peered":"2023-09-30T01:02:06.336950+0000","last_unstale":"2023-09-30T01:02:08.699728+0000","last_undegraded":"2023-09-30T01:02:08.699728+0000","last_fullsized":"2023-09-30T01:02:08.699728+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:04.607879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:04.607879+0000","last_clean_scrub_stamp":"2023-09-30T01:02:04.607879+0000","objects_scrubbed":0,"log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:29.391601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":41,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":197132,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":41,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":274444,"data_compressed":720,"data_compressed_allocated":16384,"data_compressed_original":262144,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5132,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366708,"statfs":{"total":96636764160,"available":96631508992,"internally_reserved":0,"allocated":135168,"data_stored":172126,"data_compressed":360,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5048,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366792,"statfs":{"total":96636764160,"available":96631595008,"internally_reserved":0,"allocated":114688,"data_stored":38780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5048,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366792,"statfs":{"total":96636764160,"available":96631595008,"internally_reserved":0,"allocated":114688,"data_stored":35886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5048,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366792,"statfs":{"total":96636764160,"available":96631595008,"internally_reserved":0,"allocated":114688,"data_stored":39611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5124,"kb_used_data":124,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366716,"statfs":{"total":96636764160,"available":96631517184,"internally_reserved":0,"allocated":126976,"data_stored":42886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5120,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366720,"statfs":{"total":96636764160,"available":96631521280,"internally_reserved":0,"allocated":122880,"data_stored":42272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5124,"kb_used_data":124,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366716,"statfs":{"total":96636764160,"available":96631517184,"internally_reserved":0,"allocated":126976,"data_stored":42886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":139264,"data_stored":174470,"data_compressed":360,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7450000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":135168,"data_compressed":360,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":139276,"data_compressed":360,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:02:11.194 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:02:11.469 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:11.472 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2023-09-30T01:02:11.481 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-09-30T01:02:09.590605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":197132,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":41,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":40780,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":39744,"kb_avail":754933940,"statfs":{"total":773094113280,"available":773052354560,"internally_reserved":0,"allocated":995328,"data_stored":588917,"data_compressed":720,"data_compressed_allocated":16384,"data_compressed_original":262144,"omap_allocated":0,"internal_metadata":40697856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.769880+0000","last_change":"2023-09-30T01:02:07.769880+0000","last_active":"2023-09-30T01:02:07.769880+0000","last_peered":"2023-09-30T01:02:07.769880+0000","last_clean":"2023-09-30T01:02:07.769880+0000","last_became_active":"2023-09-30T01:02:07.769178+0000","last_became_peered":"2023-09-30T01:02:07.769178+0000","last_unstale":"2023-09-30T01:02:07.769880+0000","last_undegraded":"2023-09-30T01:02:07.769880+0000","last_fullsized":"2023-09-30T01:02:07.769880+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:26.042079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.873156+0000","last_change":"2023-09-30T01:02:07.873156+0000","last_active":"2023-09-30T01:02:07.873156+0000","last_peered":"2023-09-30T01:02:07.873156+0000","last_clean":"2023-09-30T01:02:07.873156+0000","last_became_active":"2023-09-30T01:02:07.872154+0000","last_became_peered":"2023-09-30T01:02:07.872154+0000","last_unstale":"2023-09-30T01:02:07.873156+0000","last_undegraded":"2023-09-30T01:02:07.873156+0000","last_fullsized":"2023-09-30T01:02:07.873156+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:13.614779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.875706+0000","last_change":"2023-09-30T01:02:07.874353+0000","last_active":"2023-09-30T01:02:07.875706+0000","last_peered":"2023-09-30T01:02:07.875706+0000","last_clean":"2023-09-30T01:02:07.875706+0000","last_became_active":"2023-09-30T01:02:07.873496+0000","last_became_peered":"2023-09-30T01:02:07.873496+0000","last_unstale":"2023-09-30T01:02:07.875706+0000","last_undegraded":"2023-09-30T01:02:07.875706+0000","last_fullsized":"2023-09-30T01:02:07.875706+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:34.827586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.973236+0000","last_change":"2023-09-30T01:02:07.973236+0000","last_active":"2023-09-30T01:02:07.973236+0000","last_peered":"2023-09-30T01:02:07.973236+0000","last_clean":"2023-09-30T01:02:07.973236+0000","last_became_active":"2023-09-30T01:02:07.872317+0000","last_became_peered":"2023-09-30T01:02:07.872317+0000","last_unstale":"2023-09-30T01:02:07.973236+0000","last_undegraded":"2023-09-30T01:02:07.973236+0000","last_fullsized":"2023-09-30T01:02:07.973236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:14.385712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.878185+0000","last_change":"2023-09-30T01:02:07.878185+0000","last_active":"2023-09-30T01:02:07.878185+0000","last_peered":"2023-09-30T01:02:07.878185+0000","last_clean":"2023-09-30T01:02:07.878185+0000","last_became_active":"2023-09-30T01:02:07.677592+0000","last_became_peered":"2023-09-30T01:02:07.677592+0000","last_unstale":"2023-09-30T01:02:07.878185+0000","last_undegraded":"2023-09-30T01:02:07.878185+0000","last_fullsized":"2023-09-30T01:02:07.878185+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:14.385712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.775982+0000","last_change":"2023-09-30T01:02:07.775982+0000","last_active":"2023-09-30T01:02:07.775982+0000","last_peered":"2023-09-30T01:02:07.775982+0000","last_clean":"2023-09-30T01:02:07.775982+0000","last_became_active":"2023-09-30T01:02:07.775147+0000","last_became_peered":"2023-09-30T01:02:07.775147+0000","last_unstale":"2023-09-30T01:02:07.775982+0000","last_undegraded":"2023-09-30T01:02:07.775982+0000","last_fullsized":"2023-09-30T01:02:07.775982+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:27.018152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.873601+0000","last_change":"2023-09-30T01:02:07.773140+0000","last_active":"2023-09-30T01:02:07.873601+0000","last_peered":"2023-09-30T01:02:07.873601+0000","last_clean":"2023-09-30T01:02:07.873601+0000","last_became_active":"2023-09-30T01:02:07.772379+0000","last_became_peered":"2023-09-30T01:02:07.772379+0000","last_unstale":"2023-09-30T01:02:07.873601+0000","last_undegraded":"2023-09-30T01:02:07.873601+0000","last_fullsized":"2023-09-30T01:02:07.873601+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:09.142025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:07.774994+0000","last_change":"2023-09-30T01:02:07.774994+0000","last_active":"2023-09-30T01:02:07.774994+0000","last_peered":"2023-09-30T01:02:07.774994+0000","last_clean":"2023-09-30T01:02:07.774994+0000","last_became_active":"2023-09-30T01:02:07.773993+0000","last_became_peered":"2023-09-30T01:02:07.773993+0000","last_unstale":"2023-09-30T01:02:07.774994+0000","last_undegraded":"2023-09-30T01:02:07.774994+0000","last_fullsized":"2023-09-30T01:02:07.774994+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:02:06.639364+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:36.638320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'20","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T01:02:08.699728+0000","last_change":"2023-09-30T01:02:06.437512+0000","last_active":"2023-09-30T01:02:08.699728+0000","last_peered":"2023-09-30T01:02:08.699728+0000","last_clean":"2023-09-30T01:02:08.699728+0000","last_became_active":"2023-09-30T01:02:06.336950+0000","last_became_peered":"2023-09-30T01:02:06.336950+0000","last_unstale":"2023-09-30T01:02:08.699728+0000","last_undegraded":"2023-09-30T01:02:08.699728+0000","last_fullsized":"2023-09-30T01:02:08.699728+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:02:04.607879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:04.607879+0000","last_clean_scrub_stamp":"2023-09-30T01:02:04.607879+0000","objects_scrubbed":0,"log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:03:29.391601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":41,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":197132,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":41,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":274444,"data_compressed":720,"data_compressed_allocated":16384,"data_compressed_original":262144,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5132,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366708,"statfs":{"total":96636764160,"available":96631508992,"internally_reserved":0,"allocated":135168,"data_stored":172126,"data_compressed":360,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5048,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366792,"statfs":{"total":96636764160,"available":96631595008,"internally_reserved":0,"allocated":114688,"data_stored":38780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5048,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366792,"statfs":{"total":96636764160,"available":96631595008,"internally_reserved":0,"allocated":114688,"data_stored":35886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5048,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366792,"statfs":{"total":96636764160,"available":96631595008,"internally_reserved":0,"allocated":114688,"data_stored":39611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5124,"kb_used_data":124,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366716,"statfs":{"total":96636764160,"available":96631517184,"internally_reserved":0,"allocated":126976,"data_stored":42886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5120,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366720,"statfs":{"total":96636764160,"available":96631521280,"internally_reserved":0,"allocated":122880,"data_stored":42272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5124,"kb_used_data":124,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366716,"statfs":{"total":96636764160,"available":96631517184,"internally_reserved":0,"allocated":126976,"data_stored":42886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":139264,"data_stored":174470,"data_compressed":360,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7450000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":135168,"data_compressed":360,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":139276,"data_compressed":360,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:02:11.483 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-30T01:02:11.483 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-30T01:02:11.483 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-30T01:02:11.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:11.787 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:11.788 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:14.802 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:15.116 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:15.116 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:18.131 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:18.465 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:21.479 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:21.816 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:24.831 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:25.159 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:28.175 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:28.513 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:28.513 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:31.528 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:31.859 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:31.859 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:34.874 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:35.211 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:35.211 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:38.225 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:38.553 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:38.554 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:41.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:41.897 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:41.897 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:44.911 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:45.235 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:45.235 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:48.249 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:48.582 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:48.582 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:51.597 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:51.916 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:51.916 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:54.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:55.262 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:55.262 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:02:58.279 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:02:58.611 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:02:58.611 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:01.625 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:01.949 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:01.950 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:04.964 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:05.290 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:05.290 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:06.180 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-09-30T01:03:06.180+0000 7f8652074700 -1 osd.4 18 handle_osd_map injecting map crc failure 2023-09-30T01:03:08.305 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:08.628 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:11.642 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:11.978 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:14.992 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:15.330 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:15.330 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:17.310 INFO:tasks.ceph.osd.0.smithi037.stderr:2023-09-30T01:03:17.305+0000 7f8d43ca8700 -1 osd.0 20 handle_osd_map injecting map crc failure 2023-09-30T01:03:18.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:18.675 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:18.675 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:21.690 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:22.018 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:22.018 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:23.481 INFO:tasks.ceph.osd.3.smithi037.stderr:2023-09-30T01:03:23.477+0000 7f93195cb700 -1 osd.3 22 handle_osd_map injecting map crc failure 2023-09-30T01:03:25.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:25.359 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:25.359 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:28.379 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:28.705 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:30.554 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-09-30T01:03:30.551+0000 7f615e7ae700 -1 osd.7 24 handle_osd_map injecting map crc failure 2023-09-30T01:03:31.188 INFO:tasks.ceph.osd.3.smithi037.stderr:2023-09-30T01:03:31.185+0000 7f93195cb700 -1 osd.3 25 handle_osd_map injecting map crc failure 2023-09-30T01:03:31.720 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:32.050 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:32.050 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:32.051 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-09-30T01:03:32.047+0000 7f507210e700 -1 osd.6 25 handle_osd_map injecting map crc failure 2023-09-30T01:03:35.065 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:35.388 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:38.403 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:38.729 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:38.730 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:41.745 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:42.075 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:42.075 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:45.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:45.418 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:45.419 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:48.434 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:48.766 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:48.766 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:51.781 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:52.114 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:52.114 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:55.129 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:55.459 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:55.459 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:03:58.474 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:03:58.801 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:03:58.801 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:01.815 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:02.150 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:02.151 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:05.164 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:05.494 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:05.495 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:08.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:08.848 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:08.849 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:11.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:12.191 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:13.124 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-09-30T01:04:13.122+0000 7f711c629700 -1 osd.5 26 handle_osd_map injecting map crc failure 2023-09-30T01:04:13.124 INFO:tasks.ceph.osd.3.smithi037.stderr:2023-09-30T01:04:13.117+0000 7f93175c7700 -1 osd.3 26 handle_osd_map injecting map crc failure 2023-09-30T01:04:15.206 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:15.532 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:15.533 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:16.965 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-09-30T01:04:16.962+0000 7f615e7ae700 -1 osd.7 27 handle_osd_map injecting map crc failure 2023-09-30T01:04:18.548 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:18.869 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:18.869 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:21.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:22.224 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:25.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:25.578 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:25.578 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:28.593 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:28.926 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:28.926 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:31.460 INFO:tasks.ceph.osd.0.smithi037.stderr:2023-09-30T01:04:31.457+0000 7f8d45cac700 -1 osd.0 29 handle_osd_map injecting map crc failure 2023-09-30T01:04:31.940 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:32.275 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:32.275 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:35.290 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:35.626 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:35.626 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:36.001 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-09-30T01:04:35.998+0000 7f615e7ae700 -1 osd.7 30 handle_osd_map injecting map crc failure 2023-09-30T01:04:36.002 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-09-30T01:04:35.998+0000 7f507210e700 -1 osd.6 30 handle_osd_map injecting map crc failure 2023-09-30T01:04:38.641 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:38.969 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:40.865 INFO:tasks.ceph.osd.2.smithi037.stderr:2023-09-30T01:04:40.861+0000 7f5d79abe700 -1 osd.2 31 handle_osd_map injecting map crc failure 2023-09-30T01:04:41.983 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:42.319 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:42.319 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:45.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:45.668 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:45.973 INFO:tasks.ceph.osd.6.smithi050.stderr:2023-09-30T01:04:45.969+0000 7f507210e700 -1 osd.6 32 handle_osd_map injecting map crc failure 2023-09-30T01:04:46.270 INFO:tasks.ceph.osd.1.smithi037.stderr:2023-09-30T01:04:46.269+0000 7f5be8df5700 -1 osd.1 32 handle_osd_map injecting map crc failure 2023-09-30T01:04:48.683 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:49.008 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:49.008 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:52.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:52.358 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:52.358 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:55.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:55.699 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:04:58.714 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:04:59.038 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:04:59.038 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:02.053 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:02.381 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:02.381 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:05.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:05.719 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:05.719 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:08.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:09.061 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:12.076 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:12.413 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:12.414 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:15.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:15.757 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:15.757 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:18.772 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:19.102 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:19.102 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:22.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:22.450 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:22.450 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:25.465 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:25.786 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:26.275 INFO:tasks.ceph.osd.0.smithi037.stderr:2023-09-30T01:05:26.268+0000 7f8d43ca8700 -1 osd.0 34 handle_osd_map injecting map crc failure 2023-09-30T01:05:27.248 INFO:tasks.ceph.osd.1.smithi037.stderr:2023-09-30T01:05:27.244+0000 7f5be8df5700 -1 osd.1 35 handle_osd_map injecting map crc failure 2023-09-30T01:05:27.249 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-09-30T01:05:27.248+0000 7f8650070700 -1 osd.4 35 handle_osd_map injecting map crc failure 2023-09-30T01:05:28.801 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:29.133 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:29.134 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:32.147 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:32.481 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:32.481 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:35.495 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:35.839 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:38.860 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:39.187 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:42.201 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:42.528 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:42.528 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:43.422 INFO:tasks.ceph.osd.2.smithi037.stderr:2023-09-30T01:05:43.420+0000 7f5d7bac2700 -1 osd.2 36 handle_osd_map injecting map crc failure 2023-09-30T01:05:45.542 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:45.870 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:45.870 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:48.177 INFO:tasks.ceph.osd.3.smithi037.stderr:2023-09-30T01:05:48.176+0000 7f93175c7700 -1 osd.3 37 handle_osd_map injecting map crc failure 2023-09-30T01:05:48.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:49.209 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:49.209 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:52.224 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:52.552 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:52.552 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:55.567 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:55.899 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:55.899 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:05:58.914 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:05:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:05:59.245 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:01.529 INFO:tasks.ceph.osd.1.smithi037.stderr:2023-09-30T01:06:01.527+0000 7f5beadf9700 -1 osd.1 40 handle_osd_map injecting map crc failure 2023-09-30T01:06:01.529 INFO:tasks.ceph.osd.3.smithi037.stderr:2023-09-30T01:06:01.527+0000 7f93195cb700 -1 osd.3 40 handle_osd_map injecting map crc failure 2023-09-30T01:06:02.259 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:02.592 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:05.608 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:05.931 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:05.932 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:08.946 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:09.273 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:09.274 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:12.289 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:12.619 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:12.619 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:15.633 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:15.964 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:15.965 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:18.980 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:19.315 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:19.315 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:22.330 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:22.659 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:22.659 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:25.674 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:26.005 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:29.019 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:29.349 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:29.349 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:32.363 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:32.691 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:32.691 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:35.706 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:36.038 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:36.039 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:39.052 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:42.414 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:42.750 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:42.750 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:45.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:46.097 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:46.097 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:47.411 INFO:tasks.ceph.osd.2.smithi037.stderr:2023-09-30T01:06:47.406+0000 7f5d7bac2700 -1 osd.2 42 handle_osd_map injecting map crc failure 2023-09-30T01:06:47.722 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-09-30T01:06:47.718+0000 7f711c629700 -1 osd.5 42 handle_osd_map injecting map crc failure 2023-09-30T01:06:49.113 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:49.437 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:49.437 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:52.456 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:52.789 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:55.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:56.138 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:56.138 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:06:59.152 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:06:59.487 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:06:59.487 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:07:02.501 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:07:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:07:02.828 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:07:05.849 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:07:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:07:06.179 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:07:06.767 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-09-30T01:07:06.766+0000 7f615e7ae700 -1 osd.7 43 handle_osd_map injecting map crc failure 2023-09-30T01:07:09.194 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:07:09.300 INFO:tasks.ceph.osd.7.smithi050.stderr:2023-09-30T01:07:09.298+0000 7f615e7ae700 -1 osd.7 45 handle_osd_map injecting map crc failure 2023-09-30T01:07:09.524 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:07:09.524 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:07:12.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:07:12.870 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:07:12.870 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:07:12.887 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-09-30T01:07:12.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T01:07:13.178 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:07:13.178 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"fsid":"43b20d8a-eabf-4c8a-988f-eaf5c9521071","created":"2023-09-30T01:01:55.405921+0000","modified":"2023-09-30T01:07:09.531710+0000","last_up_change":"2023-09-30T01:02:04.607879+0000","last_in_change":"2023-09-30T01:01:57.341399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T01:02:03.654754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T01:02:06.017647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2e24b7d6-a497-485e-b99b-0bfa85c43338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6802","nonce":31154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6804","nonce":31154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6807","nonce":31154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6805","nonce":31154}]},"public_addr":"172.21.15.37:6802/31154","cluster_addr":"172.21.15.37:6804/31154","heartbeat_back_addr":"172.21.15.37:6807/31154","heartbeat_front_addr":"172.21.15.37:6805/31154","state":["exists","up"]},{"osd":1,"uuid":"a8f0a7a5-328d-4868-a30d-22e3b5b832ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6808","nonce":31153}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6809","nonce":31153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6811","nonce":31153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6810","nonce":31153}]},"public_addr":"172.21.15.37:6808/31153","cluster_addr":"172.21.15.37:6809/31153","heartbeat_back_addr":"172.21.15.37:6811/31153","heartbeat_front_addr":"172.21.15.37:6810/31153","state":["exists","up"]},{"osd":2,"uuid":"376a9ecf-3a2c-4bd5-add5-0d1e64bece9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6812","nonce":31155}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6813","nonce":31155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6815","nonce":31155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6814","nonce":31155}]},"public_addr":"172.21.15.37:6812/31155","cluster_addr":"172.21.15.37:6813/31155","heartbeat_back_addr":"172.21.15.37:6815/31155","heartbeat_front_addr":"172.21.15.37:6814/31155","state":["exists","up"]},{"osd":3,"uuid":"4b7b474d-a9a6-4d45-b263-da4ddeb0a5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6800","nonce":31152}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6801","nonce":31152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6806","nonce":31152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6803","nonce":31152}]},"public_addr":"172.21.15.37:6800/31152","cluster_addr":"172.21.15.37:6801/31152","heartbeat_back_addr":"172.21.15.37:6806/31152","heartbeat_front_addr":"172.21.15.37:6803/31152","state":["exists","up"]},{"osd":4,"uuid":"949fb796-6e09-474e-b21e-8ba7a52c1d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6804","nonce":31190}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6805","nonce":31190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6807","nonce":31190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6806","nonce":31190}]},"public_addr":"172.21.15.50:6804/31190","cluster_addr":"172.21.15.50:6805/31190","heartbeat_back_addr":"172.21.15.50:6807/31190","heartbeat_front_addr":"172.21.15.50:6806/31190","state":["exists","up"]},{"osd":5,"uuid":"869b70fb-be90-4d48-b9c6-a1f2d6e1682f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6800","nonce":31191}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6801","nonce":31191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6803","nonce":31191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6802","nonce":31191}]},"public_addr":"172.21.15.50:6800/31191","cluster_addr":"172.21.15.50:6801/31191","heartbeat_back_addr":"172.21.15.50:6803/31191","heartbeat_front_addr":"172.21.15.50:6802/31191","state":["exists","up"]},{"osd":6,"uuid":"bddff542-e807-42f4-91e1-c6b3db62d7a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6810","nonce":31192}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6812","nonce":31192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6815","nonce":31192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6813","nonce":31192}]},"public_addr":"172.21.15.50:6810/31192","cluster_addr":"172.21.15.50:6812/31192","heartbeat_back_addr":"172.21.15.50:6815/31192","heartbeat_front_addr":"172.21.15.50:6813/31192","state":["exists","up"]},{"osd":7,"uuid":"a1448ff4-9ea4-4fd3-a940-ed51cbb9b3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6808","nonce":31215}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6809","nonce":31215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6814","nonce":31215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6811","nonce":31215}]},"public_addr":"172.21.15.50:6808/31215","cluster_addr":"172.21.15.50:6809/31215","heartbeat_back_addr":"172.21.15.50:6814/31215","heartbeat_front_addr":"172.21.15.50:6811/31215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:07:08.000197+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:06:00.793465+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:06:47.324943+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:07:06.720443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:05:56.394856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:07:05.708097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:07:09.033957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:06:40.351536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T01:07:13.193 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T01:07:13.193 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:07:13.492 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:07:13.495 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2023-09-30T01:07:13.507 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2023-09-30T01:07:11.682084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":45572,"kb_used_data":2756,"kb_used_omap":0,"kb_used_meta":42752,"kb_avail":754929148,"statfs":{"total":773094113280,"available":773047447552,"internally_reserved":0,"allocated":2822144,"data_stored":2368309,"data_compressed":11862,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":43778048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.160387"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.906169+0000","last_change":"2023-09-30T01:06:20.261279+0000","last_active":"2023-09-30T01:07:08.906169+0000","last_peered":"2023-09-30T01:07:08.906169+0000","last_clean":"2023-09-30T01:07:08.906169+0000","last_became_active":"2023-09-30T01:02:07.769178+0000","last_became_peered":"2023-09-30T01:02:07.769178+0000","last_unstale":"2023-09-30T01:07:08.906169+0000","last_undegraded":"2023-09-30T01:07:08.906169+0000","last_fullsized":"2023-09-30T01:07:08.906169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:20.261102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:20.261102+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:36.156659+0000","scrub_duration":0.20364980799999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.104554+0000","last_change":"2023-09-30T01:06:18.345145+0000","last_active":"2023-09-30T01:07:08.104554+0000","last_peered":"2023-09-30T01:07:08.104554+0000","last_clean":"2023-09-30T01:07:08.104554+0000","last_became_active":"2023-09-30T01:02:07.872154+0000","last_became_peered":"2023-09-30T01:02:07.872154+0000","last_unstale":"2023-09-30T01:07:08.104554+0000","last_undegraded":"2023-09-30T01:07:08.104554+0000","last_fullsized":"2023-09-30T01:07:08.104554+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:18.344893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:06:18.344893+0000","last_clean_scrub_stamp":"2023-09-30T01:06:18.344893+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:22.660473+0000","scrub_duration":0.20445159399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.506898+0000","last_change":"2023-09-30T01:06:04.600417+0000","last_active":"2023-09-30T01:07:09.506898+0000","last_peered":"2023-09-30T01:07:09.506898+0000","last_clean":"2023-09-30T01:07:09.506898+0000","last_became_active":"2023-09-30T01:02:07.873496+0000","last_became_peered":"2023-09-30T01:02:07.873496+0000","last_unstale":"2023-09-30T01:07:09.506898+0000","last_undegraded":"2023-09-30T01:07:09.506898+0000","last_fullsized":"2023-09-30T01:07:09.506898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:04.600148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:03:35.046769+0000","last_clean_scrub_stamp":"2023-09-30T01:06:04.600148+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:18.895144+0000","scrub_duration":0.30469532599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":76,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.104475+0000","last_change":"2023-09-30T01:06:43.248656+0000","last_active":"2023-09-30T01:07:08.104475+0000","last_peered":"2023-09-30T01:07:08.104475+0000","last_clean":"2023-09-30T01:07:08.104475+0000","last_became_active":"2023-09-30T01:02:07.872317+0000","last_became_peered":"2023-09-30T01:02:07.872317+0000","last_unstale":"2023-09-30T01:07:08.104475+0000","last_undegraded":"2023-09-30T01:07:08.104475+0000","last_fullsized":"2023-09-30T01:07:08.104475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:43.248242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:43.248242+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:07.553083+0000","scrub_duration":0.103841286,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.107043+0000","last_change":"2023-09-30T01:06:07.319639+0000","last_active":"2023-09-30T01:07:08.107043+0000","last_peered":"2023-09-30T01:07:08.107043+0000","last_clean":"2023-09-30T01:07:08.107043+0000","last_became_active":"2023-09-30T01:02:07.677592+0000","last_became_peered":"2023-09-30T01:02:07.677592+0000","last_unstale":"2023-09-30T01:07:08.107043+0000","last_undegraded":"2023-09-30T01:07:08.107043+0000","last_fullsized":"2023-09-30T01:07:08.107043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:07.319360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:03:15.943036+0000","last_clean_scrub_stamp":"2023-09-30T01:06:07.319360+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:26.661423+0000","scrub_duration":0.304743127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:07:07.728402+0000","last_change":"2023-09-30T01:06:06.052328+0000","last_active":"2023-09-30T01:07:07.728402+0000","last_peered":"2023-09-30T01:07:07.728402+0000","last_clean":"2023-09-30T01:07:07.728402+0000","last_became_active":"2023-09-30T01:02:07.775147+0000","last_became_peered":"2023-09-30T01:02:07.775147+0000","last_unstale":"2023-09-30T01:07:07.728402+0000","last_undegraded":"2023-09-30T01:07:07.728402+0000","last_fullsized":"2023-09-30T01:07:07.728402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:06.052079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:06.052079+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:36.034258+0000","scrub_duration":0.203916498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.406857+0000","last_change":"2023-09-30T01:06:58.156271+0000","last_active":"2023-09-30T01:07:09.406857+0000","last_peered":"2023-09-30T01:07:09.406857+0000","last_clean":"2023-09-30T01:07:09.406857+0000","last_became_active":"2023-09-30T01:02:07.772379+0000","last_became_peered":"2023-09-30T01:02:07.772379+0000","last_unstale":"2023-09-30T01:07:09.406857+0000","last_undegraded":"2023-09-30T01:07:09.406857+0000","last_fullsized":"2023-09-30T01:07:09.406857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:58.156016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:06:58.156016+0000","last_clean_scrub_stamp":"2023-09-30T01:06:58.156016+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:26.133597+0000","scrub_duration":0.60520065199999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.107043+0000","last_change":"2023-09-30T01:06:15.101701+0000","last_active":"2023-09-30T01:07:08.107043+0000","last_peered":"2023-09-30T01:07:08.107043+0000","last_clean":"2023-09-30T01:07:08.107043+0000","last_became_active":"2023-09-30T01:02:07.773993+0000","last_became_peered":"2023-09-30T01:02:07.773993+0000","last_unstale":"2023-09-30T01:07:08.107043+0000","last_undegraded":"2023-09-30T01:07:08.107043+0000","last_fullsized":"2023-09-30T01:07:08.107043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:15.101414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:15.101414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:40.776761+0000","scrub_duration":0.20410309200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":220,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.406888+0000","last_change":"2023-09-30T01:06:10.629812+0000","last_active":"2023-09-30T01:07:09.406888+0000","last_peered":"2023-09-30T01:07:09.406888+0000","last_clean":"2023-09-30T01:07:09.406888+0000","last_became_active":"2023-09-30T01:02:06.336950+0000","last_became_peered":"2023-09-30T01:02:06.336950+0000","last_unstale":"2023-09-30T01:07:09.406888+0000","last_undegraded":"2023-09-30T01:07:09.406888+0000","last_fullsized":"2023-09-30T01:07:09.406888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'117","last_scrub_stamp":"2023-09-30T01:06:10.629534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:04.607879+0000","last_clean_scrub_stamp":"2023-09-30T01:06:10.629534+0000","objects_scrubbed":2,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:21.216419+0000","scrub_duration":0.30592072999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11862,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542208,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6096,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365744,"statfs":{"total":96636764160,"available":96630521856,"internally_reserved":0,"allocated":401408,"data_stored":698218,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90500000000000003,"15min":0.91500000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.73099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.306,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.95199999999999996}]},{"osd":1,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.5700000000000001,"15min":1.0880000000000001},"min":{"1min":0.65400000000000003,"5min":0.62,"15min":0.62},"max":{"1min":1.3640000000000001,"5min":78.195999999999998,"15min":78.195999999999998},"last":1.109},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.98599999999999999,"15min":1.0429999999999999},"min":{"1min":0.70499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2330000000000001,"5min":1.452,"15min":1.452},"last":0.68700000000000006}]},{"osd":2,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.92700000000000005},"min":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.99199999999999999,"15min":1.0169999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2350000000000001,"5min":1.363,"15min":1.363},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0529999999999999,"15min":1.0489999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2869999999999999,"5min":1.425,"15min":1.425},"last":1.0369999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.98299999999999998,"15min":1.0089999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.105,"15min":1.157},"min":{"1min":0.70799999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3280000000000001,"5min":1.486,"15min":1.486},"last":1.1339999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0960000000000001,"15min":1.081},"min":{"1min":0.89000000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.389,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.1830000000000001,"15min":1.1990000000000001},"min":{"1min":0.78300000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.83899999999999997}]},{"osd":6,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2529999999999999,"15min":1.2749999999999999},"min":{"1min":0.98499999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.474,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.88},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2629999999999999,"15min":1.2769999999999999},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.573,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.246}]}]},{"osd":6,"up_from":14,"seq":60129542208,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5584,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366256,"statfs":{"total":96636764160,"available":96631046144,"internally_reserved":0,"allocated":335872,"data_stored":163198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92000000000000004,"15min":0.93200000000000005},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.6659999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.29,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.74199999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.96399999999999997,"15min":0.95999999999999996},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93100000000000005,"15min":0.92600000000000005},"min":{"1min":0.68700000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.327,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.70199999999999996,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.353,"5min":1.462,"15min":1.462},"last":1.016},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.482,"15min":1.24},"min":{"1min":0.65400000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.2130000000000001,"5min":47.942,"15min":47.942},"last":0.92000000000000004}]},{"osd":3,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.038,"15min":1.0940000000000001},"min":{"1min":0.67900000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.304,"5min":1.345,"15min":1.345},"last":1.304},{"interface":"front","average":{"1min":1.075,"5min":1.2569999999999999,"15min":1.107},"min":{"1min":0.69199999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.373,"5min":20.649999999999999,"15min":20.649999999999999},"last":1.171}]},{"osd":4,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.98399999999999999,"15min":0.99199999999999999},"min":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.327,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.0369999999999999,"15min":1.046},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.298,"5min":1.405,"15min":1.405},"last":0.85399999999999998}]},{"osd":5,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.113,"15min":1.113},"min":{"1min":0.83399999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.3420000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.206,"15min":1.2569999999999999},"min":{"1min":0.93500000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.577,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.577}]},{"osd":7,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1240000000000001,"15min":1.1579999999999999},"min":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3029999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.232,"15min":1.246},"min":{"1min":0.76600000000000001,"5min":0.752,"15min":0.752},"max":{"1min":1.429,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.3680000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5532,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366308,"statfs":{"total":96636764160,"available":96631099392,"internally_reserved":0,"allocated":348160,"data_stored":165534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.90600000000000003,"15min":0.91000000000000003},"min":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.139,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.93899999999999995,"15min":0.90700000000000003},"min":{"1min":0.65300000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2709999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.72299999999999998}]},{"osd":1,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94399999999999995,"15min":0.95999999999999996},"min":{"1min":0.71399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3029999999999999,"5min":1.655,"15min":1.655},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94599999999999995,"15min":0.91800000000000004},"min":{"1min":0.70799999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2749999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.115}]},{"osd":2,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.95999999999999996,"15min":0.97799999999999998},"min":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.014,"15min":1.042},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.294,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.018,"15min":1.038},"min":{"1min":0.76300000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.236,"5min":1.431,"15min":1.431},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.73799999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3480000000000001,"5min":1.478,"15min":1.478},"last":1.4850000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.68500000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.3080000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.006},{"interface":"front","average":{"1min":1.163,"5min":1.2070000000000001,"15min":1.208},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.4119999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.583}]},{"osd":6,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0780000000000001,"15min":1.0940000000000001},"min":{"1min":0.79300000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.3009999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.1439999999999999,"15min":1.1459999999999999},"min":{"1min":0.81499999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.375,"5min":1.498,"15min":1.498},"last":1.1679999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.2490000000000001},"min":{"1min":0.997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4930000000000001,"5min":1.569,"15min":1.569},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2809999999999999,"15min":1.2769999999999999},"min":{"1min":1.075,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.5009999999999999,"5min":1.929,"15min":1.929},"last":1.4199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5584,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366256,"statfs":{"total":96636764160,"available":96631046144,"internally_reserved":0,"allocated":335872,"data_stored":164317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.92100000000000004,"15min":0.90800000000000003},"min":{"1min":0.627,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.879},{"interface":"front","average":{"1min":0.877,"5min":0.90000000000000002,"15min":0.88600000000000001},"min":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.254,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.97699999999999998}]},{"osd":1,"last update":"Sat Sep 30 01:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.228,"5min":1.387,"15min":1.387},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96899999999999997,"15min":0.95899999999999996},"min":{"1min":0.70399999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.167,"5min":1.454,"15min":1.454},"last":0.81499999999999995}]},{"osd":2,"last update":"Sat Sep 30 01:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.92500000000000004,"15min":0.89400000000000002},"min":{"1min":0.747,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.22,"5min":1.448,"15min":1.448},"last":1.093},{"interface":"front","average":{"1min":1.006,"5min":1.0609999999999999,"15min":1.0820000000000001},"min":{"1min":0.78200000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.256,"5min":1.349,"15min":1.349},"last":1.1930000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:06:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.054,"15min":1.073},"min":{"1min":0.67300000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.26,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.143},{"interface":"front","average":{"1min":1.034,"5min":1.0509999999999999,"15min":1.052},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.252,"5min":1.466,"15min":1.466},"last":0.73699999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96899999999999997,"15min":1.0069999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.119},"min":{"1min":0.71999999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.419,"5min":1.454,"15min":1.454},"last":1.0489999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.129,"15min":1.153},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3779999999999999,"5min":2.097,"15min":2.097},"last":1.258},{"interface":"front","average":{"1min":1.151,"5min":1.1459999999999999,"15min":1.163},"min":{"1min":0.90200000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.353,"5min":1.55,"15min":1.55},"last":1.23}]},{"osd":7,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2490000000000001,"15min":1.2869999999999999},"min":{"1min":0.92000000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.514,"5min":1.72,"15min":1.72},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.28,"15min":1.298},"min":{"1min":1.0600000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.472,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.3400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5512,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366328,"statfs":{"total":96636764160,"available":96631119872,"internally_reserved":0,"allocated":327680,"data_stored":158650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.77200000000000002,"15min":0.87},"min":{"1min":0.38800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89200000000000002,"5min":6.2939999999999996,"15min":6.2939999999999996},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.90700000000000003,"15min":1.0009999999999999},"min":{"1min":0.60899999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1100000000000001,"5min":6.149,"15min":6.149},"last":0.67500000000000004}]},{"osd":1,"last update":"Sat Sep 30 01:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.98599999999999999,"15min":1.27},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97799999999999998,"5min":14.689,"15min":14.689},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.875,"5min":1.159,"15min":1.427},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3069999999999999,"5min":15.281000000000001,"15min":15.281000000000001},"last":0.629}]},{"osd":2,"last update":"Sat Sep 30 01:06:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.80200000000000005,"15min":0.78500000000000003},"min":{"1min":0.58699999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92900000000000005,"5min":1.345,"15min":1.345},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.94899999999999995,"15min":0.94999999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.597,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.879}]},{"osd":4,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.0720000000000001,"15min":1.0900000000000001},"min":{"1min":0.89600000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0389999999999999,"15min":1.097},"min":{"1min":0.70899999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3939999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.208}]},{"osd":5,"last update":"Sat Sep 30 01:06:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.1140000000000001,"15min":1.169},"min":{"1min":0.752,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2829999999999999,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1830000000000001,"15min":1.2},"min":{"1min":0.81999999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.3620000000000001,"5min":1.792,"15min":1.792},"last":1.179}]},{"osd":6,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.21,"15min":1.21},"min":{"1min":0.85099999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.272,"5min":1.238,"15min":1.248},"min":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.776,"5min":1.863,"15min":1.863},"last":1.25}]},{"osd":7,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2,"15min":1.228},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.956,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.091},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2450000000000001,"15min":1.2470000000000001},"min":{"1min":0.95099999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.732,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.1499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5584,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366256,"statfs":{"total":96636764160,"available":96631046144,"internally_reserved":0,"allocated":335872,"data_stored":162650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.24,"5min":1.409,"15min":1.409},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1879999999999999,"5min":1.254,"15min":1.254},"last":1.155}]},{"osd":1,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.80800000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.069,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84199999999999997,"15min":0.85899999999999999},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0640000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.81899999999999995}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1699999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.97499999999999998,"15min":0.995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2110000000000001,"5min":2.016,"15min":2.016},"last":1.2210000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.089,"15min":1.103},"min":{"1min":0.74299999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.3089999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.264},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1040000000000001,"15min":1.1339999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.528,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.6719999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1160000000000001,"15min":1.119},"min":{"1min":0.81599999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.4910000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.351},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.159,"15min":1.1910000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.704,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.4159999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1739999999999999,"15min":1.214},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.571,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.321},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.27,"15min":1.292},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.6240000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.6299999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2869999999999999,"15min":1.2989999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.601,"5min":2.137,"15min":2.137},"last":1.736},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2669999999999999,"15min":1.296},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.5660000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":2.6539999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5592,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366248,"statfs":{"total":96636764160,"available":96631037952,"internally_reserved":0,"allocated":344064,"data_stored":163550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67900000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0469999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.012,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.622}]},{"osd":2,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.52000000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.1479999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.88800000000000001,"15min":0.90900000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.194,"5min":1.726,"15min":1.726},"last":0.75700000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90100000000000002,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3420000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.91100000000000003},"min":{"1min":0.64000000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2210000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.72099999999999997}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.1080000000000001,"15min":1.1499999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.411,"5min":1.893,"15min":1.893},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.127,"15min":1.133},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.466,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.256}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1140000000000001,"15min":1.1240000000000001},"min":{"1min":0.91100000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5189999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.302},{"interface":"front","average":{"1min":1.085,"5min":1.117,"15min":1.1319999999999999},"min":{"1min":0.73099999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4450000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.99199999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.143,"15min":1.167},"min":{"1min":0.84999999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.4019999999999999,"5min":1.508,"15min":1.508},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.226,"15min":1.25},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.583,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.0669999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.282,"15min":1.3109999999999999},"min":{"1min":0.98599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.669,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.2},{"interface":"front","average":{"1min":1.198,"5min":1.252,"15min":1.2929999999999999},"min":{"1min":0.98399999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.5469999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.0920000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6088,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365752,"statfs":{"total":96636764160,"available":96630530048,"internally_reserved":0,"allocated":393216,"data_stored":692192,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68100000000000005,"15min":0.70699999999999996},"min":{"1min":0.53800000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.80500000000000005,"15min":0.81699999999999995},"min":{"1min":0.55800000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.208,"5min":1.377,"15min":1.377},"last":0.98299999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.121,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.746,"15min":0.745},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.2370000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.75800000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.63,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0429999999999999,"5min":1.075,"15min":1.075},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.999,"5min":0.96999999999999997,"15min":0.98699999999999999},"min":{"1min":0.70899999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.018,"15min":1.0489999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.3879999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.101,"15min":1.1359999999999999},"min":{"1min":0.91300000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4930000000000001,"5min":1.655,"15min":1.655},"last":1.1479999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0820000000000001,"15min":1.1060000000000001},"min":{"1min":0.79600000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.476,"5min":1.482,"15min":1.482},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.1850000000000001,"15min":1.208},"min":{"1min":0.94999999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4610000000000001,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.0940000000000001}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.133,"15min":1.139},"min":{"1min":0.79900000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.226,"15min":1.26},"min":{"1min":0.98199999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.1899999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2549999999999999,"15min":1.3029999999999999},"min":{"1min":1.0429999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.5720000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.375},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.224,"15min":1.25},"min":{"1min":0.89600000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.6679999999999999,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.0629999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:07:13.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:07:13.801 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:07:13.803 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2023-09-30T01:07:13.815 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2023-09-30T01:07:13.682687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":45668,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":42816,"kb_avail":754929052,"statfs":{"total":773094113280,"available":773047349248,"internally_reserved":0,"allocated":2854912,"data_stored":2385621,"data_compressed":11862,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":43843584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.155501"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.906169+0000","last_change":"2023-09-30T01:06:20.261279+0000","last_active":"2023-09-30T01:07:08.906169+0000","last_peered":"2023-09-30T01:07:08.906169+0000","last_clean":"2023-09-30T01:07:08.906169+0000","last_became_active":"2023-09-30T01:02:07.769178+0000","last_became_peered":"2023-09-30T01:02:07.769178+0000","last_unstale":"2023-09-30T01:07:08.906169+0000","last_undegraded":"2023-09-30T01:07:08.906169+0000","last_fullsized":"2023-09-30T01:07:08.906169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:20.261102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:20.261102+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:36.156659+0000","scrub_duration":0.20364980799999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.104554+0000","last_change":"2023-09-30T01:06:18.345145+0000","last_active":"2023-09-30T01:07:08.104554+0000","last_peered":"2023-09-30T01:07:08.104554+0000","last_clean":"2023-09-30T01:07:08.104554+0000","last_became_active":"2023-09-30T01:02:07.872154+0000","last_became_peered":"2023-09-30T01:02:07.872154+0000","last_unstale":"2023-09-30T01:07:08.104554+0000","last_undegraded":"2023-09-30T01:07:08.104554+0000","last_fullsized":"2023-09-30T01:07:08.104554+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:18.344893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:06:18.344893+0000","last_clean_scrub_stamp":"2023-09-30T01:06:18.344893+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:22.660473+0000","scrub_duration":0.20445159399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.506898+0000","last_change":"2023-09-30T01:06:04.600417+0000","last_active":"2023-09-30T01:07:09.506898+0000","last_peered":"2023-09-30T01:07:09.506898+0000","last_clean":"2023-09-30T01:07:09.506898+0000","last_became_active":"2023-09-30T01:02:07.873496+0000","last_became_peered":"2023-09-30T01:02:07.873496+0000","last_unstale":"2023-09-30T01:07:09.506898+0000","last_undegraded":"2023-09-30T01:07:09.506898+0000","last_fullsized":"2023-09-30T01:07:09.506898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:04.600148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:03:35.046769+0000","last_clean_scrub_stamp":"2023-09-30T01:06:04.600148+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:18.895144+0000","scrub_duration":0.30469532599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":76,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.104475+0000","last_change":"2023-09-30T01:06:43.248656+0000","last_active":"2023-09-30T01:07:08.104475+0000","last_peered":"2023-09-30T01:07:08.104475+0000","last_clean":"2023-09-30T01:07:08.104475+0000","last_became_active":"2023-09-30T01:02:07.872317+0000","last_became_peered":"2023-09-30T01:02:07.872317+0000","last_unstale":"2023-09-30T01:07:08.104475+0000","last_undegraded":"2023-09-30T01:07:08.104475+0000","last_fullsized":"2023-09-30T01:07:08.104475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:43.248242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:43.248242+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:07.553083+0000","scrub_duration":0.103841286,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.538172+0000","last_change":"2023-09-30T01:06:07.319639+0000","last_active":"2023-09-30T01:07:08.538172+0000","last_peered":"2023-09-30T01:07:08.538172+0000","last_clean":"2023-09-30T01:07:08.538172+0000","last_became_active":"2023-09-30T01:02:07.677592+0000","last_became_peered":"2023-09-30T01:02:07.677592+0000","last_unstale":"2023-09-30T01:07:08.538172+0000","last_undegraded":"2023-09-30T01:07:08.538172+0000","last_fullsized":"2023-09-30T01:07:08.538172+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:07.319360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:03:15.943036+0000","last_clean_scrub_stamp":"2023-09-30T01:06:07.319360+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:26.661423+0000","scrub_duration":0.304743127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:07:07.728402+0000","last_change":"2023-09-30T01:06:06.052328+0000","last_active":"2023-09-30T01:07:07.728402+0000","last_peered":"2023-09-30T01:07:07.728402+0000","last_clean":"2023-09-30T01:07:07.728402+0000","last_became_active":"2023-09-30T01:02:07.775147+0000","last_became_peered":"2023-09-30T01:02:07.775147+0000","last_unstale":"2023-09-30T01:07:07.728402+0000","last_undegraded":"2023-09-30T01:07:07.728402+0000","last_fullsized":"2023-09-30T01:07:07.728402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:06.052079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:06.052079+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:36.034258+0000","scrub_duration":0.203916498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.406857+0000","last_change":"2023-09-30T01:06:58.156271+0000","last_active":"2023-09-30T01:07:09.406857+0000","last_peered":"2023-09-30T01:07:09.406857+0000","last_clean":"2023-09-30T01:07:09.406857+0000","last_became_active":"2023-09-30T01:02:07.772379+0000","last_became_peered":"2023-09-30T01:02:07.772379+0000","last_unstale":"2023-09-30T01:07:09.406857+0000","last_undegraded":"2023-09-30T01:07:09.406857+0000","last_fullsized":"2023-09-30T01:07:09.406857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:58.156016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:06:58.156016+0000","last_clean_scrub_stamp":"2023-09-30T01:06:58.156016+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:26.133597+0000","scrub_duration":0.60520065199999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.538151+0000","last_change":"2023-09-30T01:06:15.101701+0000","last_active":"2023-09-30T01:07:08.538151+0000","last_peered":"2023-09-30T01:07:08.538151+0000","last_clean":"2023-09-30T01:07:08.538151+0000","last_became_active":"2023-09-30T01:02:07.773993+0000","last_became_peered":"2023-09-30T01:02:07.773993+0000","last_unstale":"2023-09-30T01:07:08.538151+0000","last_undegraded":"2023-09-30T01:07:08.538151+0000","last_fullsized":"2023-09-30T01:07:08.538151+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:15.101414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:15.101414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:40.776761+0000","scrub_duration":0.20410309200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":220,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.406888+0000","last_change":"2023-09-30T01:06:10.629812+0000","last_active":"2023-09-30T01:07:09.406888+0000","last_peered":"2023-09-30T01:07:09.406888+0000","last_clean":"2023-09-30T01:07:09.406888+0000","last_became_active":"2023-09-30T01:02:06.336950+0000","last_became_peered":"2023-09-30T01:02:06.336950+0000","last_unstale":"2023-09-30T01:07:09.406888+0000","last_undegraded":"2023-09-30T01:07:09.406888+0000","last_fullsized":"2023-09-30T01:07:09.406888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'117","last_scrub_stamp":"2023-09-30T01:06:10.629534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:04.607879+0000","last_clean_scrub_stamp":"2023-09-30T01:06:10.629534+0000","objects_scrubbed":2,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:21.216419+0000","scrub_duration":0.30592072999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11862,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542208,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6096,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365744,"statfs":{"total":96636764160,"available":96630521856,"internally_reserved":0,"allocated":401408,"data_stored":698218,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90500000000000003,"15min":0.91500000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.73099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.306,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.95199999999999996}]},{"osd":1,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.5700000000000001,"15min":1.0880000000000001},"min":{"1min":0.65400000000000003,"5min":0.62,"15min":0.62},"max":{"1min":1.3640000000000001,"5min":78.195999999999998,"15min":78.195999999999998},"last":1.109},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.98599999999999999,"15min":1.0429999999999999},"min":{"1min":0.70499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2330000000000001,"5min":1.452,"15min":1.452},"last":0.68700000000000006}]},{"osd":2,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.92700000000000005},"min":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.99199999999999999,"15min":1.0169999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2350000000000001,"5min":1.363,"15min":1.363},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0529999999999999,"15min":1.0489999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2869999999999999,"5min":1.425,"15min":1.425},"last":1.0369999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.98299999999999998,"15min":1.0089999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.105,"15min":1.157},"min":{"1min":0.70799999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3280000000000001,"5min":1.486,"15min":1.486},"last":1.1339999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0960000000000001,"15min":1.081},"min":{"1min":0.89000000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.389,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.1830000000000001,"15min":1.1990000000000001},"min":{"1min":0.78300000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.83899999999999997}]},{"osd":6,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2529999999999999,"15min":1.2749999999999999},"min":{"1min":0.98499999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.474,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.88},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2629999999999999,"15min":1.2769999999999999},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.573,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.246}]}]},{"osd":6,"up_from":14,"seq":60129542208,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5584,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366256,"statfs":{"total":96636764160,"available":96631046144,"internally_reserved":0,"allocated":335872,"data_stored":163198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92000000000000004,"15min":0.93200000000000005},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.6659999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.29,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.74199999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.96399999999999997,"15min":0.95999999999999996},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93100000000000005,"15min":0.92600000000000005},"min":{"1min":0.68700000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.327,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.70199999999999996,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.353,"5min":1.462,"15min":1.462},"last":1.016},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.482,"15min":1.24},"min":{"1min":0.65400000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.2130000000000001,"5min":47.942,"15min":47.942},"last":0.92000000000000004}]},{"osd":3,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.038,"15min":1.0940000000000001},"min":{"1min":0.67900000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.304,"5min":1.345,"15min":1.345},"last":1.304},{"interface":"front","average":{"1min":1.075,"5min":1.2569999999999999,"15min":1.107},"min":{"1min":0.69199999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.373,"5min":20.649999999999999,"15min":20.649999999999999},"last":1.171}]},{"osd":4,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.98399999999999999,"15min":0.99199999999999999},"min":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.327,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.0369999999999999,"15min":1.046},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.298,"5min":1.405,"15min":1.405},"last":0.85399999999999998}]},{"osd":5,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.113,"15min":1.113},"min":{"1min":0.83399999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.3420000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.206,"15min":1.2569999999999999},"min":{"1min":0.93500000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.577,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.577}]},{"osd":7,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1240000000000001,"15min":1.1579999999999999},"min":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3029999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.232,"15min":1.246},"min":{"1min":0.76600000000000001,"5min":0.752,"15min":0.752},"max":{"1min":1.429,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.3680000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5532,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366308,"statfs":{"total":96636764160,"available":96631099392,"internally_reserved":0,"allocated":348160,"data_stored":165534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.90600000000000003,"15min":0.91000000000000003},"min":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.139,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.93899999999999995,"15min":0.90700000000000003},"min":{"1min":0.65300000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2709999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.72299999999999998}]},{"osd":1,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94399999999999995,"15min":0.95999999999999996},"min":{"1min":0.71399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3029999999999999,"5min":1.655,"15min":1.655},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94599999999999995,"15min":0.91800000000000004},"min":{"1min":0.70799999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2749999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.115}]},{"osd":2,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.95999999999999996,"15min":0.97799999999999998},"min":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.014,"15min":1.042},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.294,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.018,"15min":1.038},"min":{"1min":0.76300000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.236,"5min":1.431,"15min":1.431},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.73799999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3480000000000001,"5min":1.478,"15min":1.478},"last":1.4850000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.68500000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.3080000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.006},{"interface":"front","average":{"1min":1.163,"5min":1.2070000000000001,"15min":1.208},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.4119999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.583}]},{"osd":6,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0780000000000001,"15min":1.0940000000000001},"min":{"1min":0.79300000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.3009999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.1439999999999999,"15min":1.1459999999999999},"min":{"1min":0.81499999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.375,"5min":1.498,"15min":1.498},"last":1.1679999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.2490000000000001},"min":{"1min":0.997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4930000000000001,"5min":1.569,"15min":1.569},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2809999999999999,"15min":1.2769999999999999},"min":{"1min":1.075,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.5009999999999999,"5min":1.929,"15min":1.929},"last":1.4199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5592,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366248,"statfs":{"total":96636764160,"available":96631037952,"internally_reserved":0,"allocated":344064,"data_stored":168645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.93100000000000005,"15min":0.91100000000000003},"min":{"1min":0.67900000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.228,"5min":1.288,"15min":1.288},"last":0.998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.91800000000000004,"15min":0.89200000000000002},"min":{"1min":0.60799999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2470000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.038}]},{"osd":1,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.94599999999999995,"15min":0.97599999999999998},"min":{"1min":0.61799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3280000000000001,"5min":1.387,"15min":1.387},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.995,"5min":0.97699999999999998,"15min":0.96199999999999997},"min":{"1min":0.70699999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2889999999999999,"5min":1.454,"15min":1.454},"last":0.60799999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93799999999999994,"15min":0.89800000000000002},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1890000000000001,"5min":1.448,"15min":1.448},"last":1.085},{"interface":"front","average":{"1min":0.996,"5min":1.042,"15min":1.0760000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.264,"5min":1.349,"15min":1.349},"last":1.139}]},{"osd":3,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.052,"15min":1.0720000000000001},"min":{"1min":0.70999999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.448,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.071,"5min":1.0549999999999999,"15min":1.0529999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.3720000000000001,"5min":1.466,"15min":1.466},"last":1.2290000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96899999999999997,"15min":1.0069999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.119},"min":{"1min":0.71999999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.419,"5min":1.454,"15min":1.454},"last":1.325}]},{"osd":6,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.129,"15min":1.153},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3779999999999999,"5min":2.097,"15min":2.097},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.1459999999999999,"15min":1.163},"min":{"1min":0.90200000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.353,"5min":1.55,"15min":1.55},"last":1.341}]},{"osd":7,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2490000000000001,"15min":1.2869999999999999},"min":{"1min":0.92000000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.514,"5min":1.72,"15min":1.72},"last":1.274},{"interface":"front","average":{"1min":1.278,"5min":1.28,"15min":1.298},"min":{"1min":1.0600000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.472,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.296}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5600,"kb_used_data":344,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366240,"statfs":{"total":96636764160,"available":96631029760,"internally_reserved":0,"allocated":352256,"data_stored":171634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.85399999999999998},"min":{"1min":0.55200000000000005,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1619999999999999,"5min":6.2939999999999996,"15min":6.2939999999999996},"last":0.627},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.84299999999999997,"15min":0.97999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.99099999999999999,"5min":6.149,"15min":6.149},"last":0.80300000000000005}]},{"osd":1,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.84999999999999998,"15min":1.2250000000000001},"min":{"1min":0.57799999999999996,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0309999999999999,"5min":14.689,"15min":14.689},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":1.014,"15min":1.379},"min":{"1min":0.629,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.081,"5min":15.281000000000001,"15min":15.281000000000001},"last":0.56399999999999995}]},{"osd":2,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81200000000000006,"15min":0.78800000000000003},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.006,"5min":1.345,"15min":1.345},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93999999999999995,"15min":0.94699999999999995},"min":{"1min":0.66200000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.2569999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.0720000000000001,"15min":1.0900000000000001},"min":{"1min":0.89600000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0389999999999999,"15min":1.097},"min":{"1min":0.70899999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3939999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.83699999999999997}]},{"osd":5,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.0960000000000001,"15min":1.163},"min":{"1min":0.89600000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.3300000000000001,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.165,"15min":1.194},"min":{"1min":0.89300000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.369,"5min":1.792,"15min":1.792},"last":0.96299999999999997}]},{"osd":6,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.21,"15min":1.21},"min":{"1min":0.85099999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.238,"15min":1.248},"min":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.776,"5min":1.863,"15min":1.863},"last":0.85399999999999998}]},{"osd":7,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2,"15min":1.228},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.956,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2450000000000001,"15min":1.2470000000000001},"min":{"1min":0.95099999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.732,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.874}]}]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5584,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366256,"statfs":{"total":96636764160,"available":96631046144,"internally_reserved":0,"allocated":335872,"data_stored":162650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.24,"5min":1.409,"15min":1.409},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1879999999999999,"5min":1.254,"15min":1.254},"last":1.155}]},{"osd":1,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.80800000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.069,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84199999999999997,"15min":0.85899999999999999},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0640000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.81899999999999995}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1699999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.97499999999999998,"15min":0.995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2110000000000001,"5min":2.016,"15min":2.016},"last":1.2210000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.089,"15min":1.103},"min":{"1min":0.74299999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.3089999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.264},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1040000000000001,"15min":1.1339999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.528,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.6719999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1160000000000001,"15min":1.119},"min":{"1min":0.81599999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.4910000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.351},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.159,"15min":1.1910000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.704,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.4159999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1739999999999999,"15min":1.214},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.571,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.321},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.27,"15min":1.292},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.6240000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.6299999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2869999999999999,"15min":1.2989999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.601,"5min":2.137,"15min":2.137},"last":1.736},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2669999999999999,"15min":1.296},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.5660000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":2.6539999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5592,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366248,"statfs":{"total":96636764160,"available":96631037952,"internally_reserved":0,"allocated":344064,"data_stored":163550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67900000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0469999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.012,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.622}]},{"osd":2,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.52000000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.1479999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.88800000000000001,"15min":0.90900000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.194,"5min":1.726,"15min":1.726},"last":0.75700000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90100000000000002,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3420000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.91100000000000003},"min":{"1min":0.64000000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2210000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.72099999999999997}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.1080000000000001,"15min":1.1499999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.411,"5min":1.893,"15min":1.893},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.127,"15min":1.133},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.466,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.256}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1140000000000001,"15min":1.1240000000000001},"min":{"1min":0.91100000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5189999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.302},{"interface":"front","average":{"1min":1.085,"5min":1.117,"15min":1.1319999999999999},"min":{"1min":0.73099999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4450000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.99199999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.143,"15min":1.167},"min":{"1min":0.84999999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.4019999999999999,"5min":1.508,"15min":1.508},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.226,"15min":1.25},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.583,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.0669999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.282,"15min":1.3109999999999999},"min":{"1min":0.98599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.669,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.2},{"interface":"front","average":{"1min":1.198,"5min":1.252,"15min":1.2929999999999999},"min":{"1min":0.98399999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.5469999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.0920000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6088,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365752,"statfs":{"total":96636764160,"available":96630530048,"internally_reserved":0,"allocated":393216,"data_stored":692192,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68100000000000005,"15min":0.70699999999999996},"min":{"1min":0.53800000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.80500000000000005,"15min":0.81699999999999995},"min":{"1min":0.55800000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.208,"5min":1.377,"15min":1.377},"last":0.98299999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.121,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.746,"15min":0.745},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.2370000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.75800000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.63,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0429999999999999,"5min":1.075,"15min":1.075},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.999,"5min":0.96999999999999997,"15min":0.98699999999999999},"min":{"1min":0.70899999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.018,"15min":1.0489999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.3879999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.101,"15min":1.1359999999999999},"min":{"1min":0.91300000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4930000000000001,"5min":1.655,"15min":1.655},"last":1.1479999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0820000000000001,"15min":1.1060000000000001},"min":{"1min":0.79600000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.476,"5min":1.482,"15min":1.482},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.1850000000000001,"15min":1.208},"min":{"1min":0.94999999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4610000000000001,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.0940000000000001}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.133,"15min":1.139},"min":{"1min":0.79900000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.226,"15min":1.26},"min":{"1min":0.98199999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.1899999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2549999999999999,"15min":1.3029999999999999},"min":{"1min":1.0429999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.5720000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.375},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.224,"15min":1.25},"min":{"1min":0.89600000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.6679999999999999,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.0629999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:07:13.817 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-30T01:07:13.818 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:07:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:07:14.102 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2023-09-30T01:07:14.114 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2023-09-30T01:07:13.682687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":45668,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":42816,"kb_avail":754929052,"statfs":{"total":773094113280,"available":773047349248,"internally_reserved":0,"allocated":2854912,"data_stored":2385621,"data_compressed":11862,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":43843584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.155501"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.906169+0000","last_change":"2023-09-30T01:06:20.261279+0000","last_active":"2023-09-30T01:07:08.906169+0000","last_peered":"2023-09-30T01:07:08.906169+0000","last_clean":"2023-09-30T01:07:08.906169+0000","last_became_active":"2023-09-30T01:02:07.769178+0000","last_became_peered":"2023-09-30T01:02:07.769178+0000","last_unstale":"2023-09-30T01:07:08.906169+0000","last_undegraded":"2023-09-30T01:07:08.906169+0000","last_fullsized":"2023-09-30T01:07:08.906169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:20.261102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:20.261102+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:36.156659+0000","scrub_duration":0.20364980799999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.104554+0000","last_change":"2023-09-30T01:06:18.345145+0000","last_active":"2023-09-30T01:07:08.104554+0000","last_peered":"2023-09-30T01:07:08.104554+0000","last_clean":"2023-09-30T01:07:08.104554+0000","last_became_active":"2023-09-30T01:02:07.872154+0000","last_became_peered":"2023-09-30T01:02:07.872154+0000","last_unstale":"2023-09-30T01:07:08.104554+0000","last_undegraded":"2023-09-30T01:07:08.104554+0000","last_fullsized":"2023-09-30T01:07:08.104554+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:18.344893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:06:18.344893+0000","last_clean_scrub_stamp":"2023-09-30T01:06:18.344893+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:22.660473+0000","scrub_duration":0.20445159399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.506898+0000","last_change":"2023-09-30T01:06:04.600417+0000","last_active":"2023-09-30T01:07:09.506898+0000","last_peered":"2023-09-30T01:07:09.506898+0000","last_clean":"2023-09-30T01:07:09.506898+0000","last_became_active":"2023-09-30T01:02:07.873496+0000","last_became_peered":"2023-09-30T01:02:07.873496+0000","last_unstale":"2023-09-30T01:07:09.506898+0000","last_undegraded":"2023-09-30T01:07:09.506898+0000","last_fullsized":"2023-09-30T01:07:09.506898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:04.600148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:03:35.046769+0000","last_clean_scrub_stamp":"2023-09-30T01:06:04.600148+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:18.895144+0000","scrub_duration":0.30469532599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":76,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.104475+0000","last_change":"2023-09-30T01:06:43.248656+0000","last_active":"2023-09-30T01:07:08.104475+0000","last_peered":"2023-09-30T01:07:08.104475+0000","last_clean":"2023-09-30T01:07:08.104475+0000","last_became_active":"2023-09-30T01:02:07.872317+0000","last_became_peered":"2023-09-30T01:02:07.872317+0000","last_unstale":"2023-09-30T01:07:08.104475+0000","last_undegraded":"2023-09-30T01:07:08.104475+0000","last_fullsized":"2023-09-30T01:07:08.104475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:43.248242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:43.248242+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:07.553083+0000","scrub_duration":0.103841286,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.538172+0000","last_change":"2023-09-30T01:06:07.319639+0000","last_active":"2023-09-30T01:07:08.538172+0000","last_peered":"2023-09-30T01:07:08.538172+0000","last_clean":"2023-09-30T01:07:08.538172+0000","last_became_active":"2023-09-30T01:02:07.677592+0000","last_became_peered":"2023-09-30T01:02:07.677592+0000","last_unstale":"2023-09-30T01:07:08.538172+0000","last_undegraded":"2023-09-30T01:07:08.538172+0000","last_fullsized":"2023-09-30T01:07:08.538172+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:07.319360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:03:15.943036+0000","last_clean_scrub_stamp":"2023-09-30T01:06:07.319360+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:26.661423+0000","scrub_duration":0.304743127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:07:07.728402+0000","last_change":"2023-09-30T01:06:06.052328+0000","last_active":"2023-09-30T01:07:07.728402+0000","last_peered":"2023-09-30T01:07:07.728402+0000","last_clean":"2023-09-30T01:07:07.728402+0000","last_became_active":"2023-09-30T01:02:07.775147+0000","last_became_peered":"2023-09-30T01:02:07.775147+0000","last_unstale":"2023-09-30T01:07:07.728402+0000","last_undegraded":"2023-09-30T01:07:07.728402+0000","last_fullsized":"2023-09-30T01:07:07.728402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:06.052079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:06.052079+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:36.034258+0000","scrub_duration":0.203916498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.406857+0000","last_change":"2023-09-30T01:06:58.156271+0000","last_active":"2023-09-30T01:07:09.406857+0000","last_peered":"2023-09-30T01:07:09.406857+0000","last_clean":"2023-09-30T01:07:09.406857+0000","last_became_active":"2023-09-30T01:02:07.772379+0000","last_became_peered":"2023-09-30T01:02:07.772379+0000","last_unstale":"2023-09-30T01:07:09.406857+0000","last_undegraded":"2023-09-30T01:07:09.406857+0000","last_fullsized":"2023-09-30T01:07:09.406857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:58.156016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:06:58.156016+0000","last_clean_scrub_stamp":"2023-09-30T01:06:58.156016+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:26.133597+0000","scrub_duration":0.60520065199999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.538151+0000","last_change":"2023-09-30T01:06:15.101701+0000","last_active":"2023-09-30T01:07:08.538151+0000","last_peered":"2023-09-30T01:07:08.538151+0000","last_clean":"2023-09-30T01:07:08.538151+0000","last_became_active":"2023-09-30T01:02:07.773993+0000","last_became_peered":"2023-09-30T01:02:07.773993+0000","last_unstale":"2023-09-30T01:07:08.538151+0000","last_undegraded":"2023-09-30T01:07:08.538151+0000","last_fullsized":"2023-09-30T01:07:08.538151+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:15.101414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:15.101414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:40.776761+0000","scrub_duration":0.20410309200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":220,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.406888+0000","last_change":"2023-09-30T01:06:10.629812+0000","last_active":"2023-09-30T01:07:09.406888+0000","last_peered":"2023-09-30T01:07:09.406888+0000","last_clean":"2023-09-30T01:07:09.406888+0000","last_became_active":"2023-09-30T01:02:06.336950+0000","last_became_peered":"2023-09-30T01:02:06.336950+0000","last_unstale":"2023-09-30T01:07:09.406888+0000","last_undegraded":"2023-09-30T01:07:09.406888+0000","last_fullsized":"2023-09-30T01:07:09.406888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'117","last_scrub_stamp":"2023-09-30T01:06:10.629534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:04.607879+0000","last_clean_scrub_stamp":"2023-09-30T01:06:10.629534+0000","objects_scrubbed":2,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:21.216419+0000","scrub_duration":0.30592072999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11862,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542208,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6096,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365744,"statfs":{"total":96636764160,"available":96630521856,"internally_reserved":0,"allocated":401408,"data_stored":698218,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90500000000000003,"15min":0.91500000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.73099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.306,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.95199999999999996}]},{"osd":1,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.5700000000000001,"15min":1.0880000000000001},"min":{"1min":0.65400000000000003,"5min":0.62,"15min":0.62},"max":{"1min":1.3640000000000001,"5min":78.195999999999998,"15min":78.195999999999998},"last":1.109},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.98599999999999999,"15min":1.0429999999999999},"min":{"1min":0.70499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2330000000000001,"5min":1.452,"15min":1.452},"last":0.68700000000000006}]},{"osd":2,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.92700000000000005},"min":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.99199999999999999,"15min":1.0169999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2350000000000001,"5min":1.363,"15min":1.363},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0529999999999999,"15min":1.0489999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2869999999999999,"5min":1.425,"15min":1.425},"last":1.0369999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.98299999999999998,"15min":1.0089999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.105,"15min":1.157},"min":{"1min":0.70799999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3280000000000001,"5min":1.486,"15min":1.486},"last":1.1339999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0960000000000001,"15min":1.081},"min":{"1min":0.89000000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.389,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.1830000000000001,"15min":1.1990000000000001},"min":{"1min":0.78300000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.83899999999999997}]},{"osd":6,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2529999999999999,"15min":1.2749999999999999},"min":{"1min":0.98499999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.474,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.88},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2629999999999999,"15min":1.2769999999999999},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.573,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.246}]}]},{"osd":6,"up_from":14,"seq":60129542208,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5584,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366256,"statfs":{"total":96636764160,"available":96631046144,"internally_reserved":0,"allocated":335872,"data_stored":163198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92000000000000004,"15min":0.93200000000000005},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.6659999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.29,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.74199999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.96399999999999997,"15min":0.95999999999999996},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93100000000000005,"15min":0.92600000000000005},"min":{"1min":0.68700000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.327,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.70199999999999996,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.353,"5min":1.462,"15min":1.462},"last":1.016},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.482,"15min":1.24},"min":{"1min":0.65400000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.2130000000000001,"5min":47.942,"15min":47.942},"last":0.92000000000000004}]},{"osd":3,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.038,"15min":1.0940000000000001},"min":{"1min":0.67900000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.304,"5min":1.345,"15min":1.345},"last":1.304},{"interface":"front","average":{"1min":1.075,"5min":1.2569999999999999,"15min":1.107},"min":{"1min":0.69199999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.373,"5min":20.649999999999999,"15min":20.649999999999999},"last":1.171}]},{"osd":4,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.98399999999999999,"15min":0.99199999999999999},"min":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.327,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.0369999999999999,"15min":1.046},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.298,"5min":1.405,"15min":1.405},"last":0.85399999999999998}]},{"osd":5,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.113,"15min":1.113},"min":{"1min":0.83399999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.3420000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.206,"15min":1.2569999999999999},"min":{"1min":0.93500000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.577,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.577}]},{"osd":7,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1240000000000001,"15min":1.1579999999999999},"min":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3029999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.232,"15min":1.246},"min":{"1min":0.76600000000000001,"5min":0.752,"15min":0.752},"max":{"1min":1.429,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.3680000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5532,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366308,"statfs":{"total":96636764160,"available":96631099392,"internally_reserved":0,"allocated":348160,"data_stored":165534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.90600000000000003,"15min":0.91000000000000003},"min":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.139,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.93899999999999995,"15min":0.90700000000000003},"min":{"1min":0.65300000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2709999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.72299999999999998}]},{"osd":1,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94399999999999995,"15min":0.95999999999999996},"min":{"1min":0.71399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3029999999999999,"5min":1.655,"15min":1.655},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94599999999999995,"15min":0.91800000000000004},"min":{"1min":0.70799999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2749999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.115}]},{"osd":2,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.95999999999999996,"15min":0.97799999999999998},"min":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.014,"15min":1.042},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.294,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.018,"15min":1.038},"min":{"1min":0.76300000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.236,"5min":1.431,"15min":1.431},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.73799999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3480000000000001,"5min":1.478,"15min":1.478},"last":1.4850000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94899999999999995,"15min":0.93000000000000005},"min":{"1min":0.68500000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.3080000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.006},{"interface":"front","average":{"1min":1.163,"5min":1.2070000000000001,"15min":1.208},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.4119999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.583}]},{"osd":6,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0780000000000001,"15min":1.0940000000000001},"min":{"1min":0.79300000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.3009999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.1439999999999999,"15min":1.1459999999999999},"min":{"1min":0.81499999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.375,"5min":1.498,"15min":1.498},"last":1.1679999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.2490000000000001},"min":{"1min":0.997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4930000000000001,"5min":1.569,"15min":1.569},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2809999999999999,"15min":1.2769999999999999},"min":{"1min":1.075,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.5009999999999999,"5min":1.929,"15min":1.929},"last":1.4199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5592,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366248,"statfs":{"total":96636764160,"available":96631037952,"internally_reserved":0,"allocated":344064,"data_stored":168645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.93100000000000005,"15min":0.91100000000000003},"min":{"1min":0.67900000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.228,"5min":1.288,"15min":1.288},"last":0.998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.91800000000000004,"15min":0.89200000000000002},"min":{"1min":0.60799999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2470000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.038}]},{"osd":1,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.94599999999999995,"15min":0.97599999999999998},"min":{"1min":0.61799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3280000000000001,"5min":1.387,"15min":1.387},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.995,"5min":0.97699999999999998,"15min":0.96199999999999997},"min":{"1min":0.70699999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2889999999999999,"5min":1.454,"15min":1.454},"last":0.60799999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93799999999999994,"15min":0.89800000000000002},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1890000000000001,"5min":1.448,"15min":1.448},"last":1.085},{"interface":"front","average":{"1min":0.996,"5min":1.042,"15min":1.0760000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.264,"5min":1.349,"15min":1.349},"last":1.139}]},{"osd":3,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.052,"15min":1.0720000000000001},"min":{"1min":0.70999999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.448,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.071,"5min":1.0549999999999999,"15min":1.0529999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.3720000000000001,"5min":1.466,"15min":1.466},"last":1.2290000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96899999999999997,"15min":1.0069999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.119},"min":{"1min":0.71999999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.419,"5min":1.454,"15min":1.454},"last":1.325}]},{"osd":6,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.129,"15min":1.153},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3779999999999999,"5min":2.097,"15min":2.097},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.1459999999999999,"15min":1.163},"min":{"1min":0.90200000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.353,"5min":1.55,"15min":1.55},"last":1.341}]},{"osd":7,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2490000000000001,"15min":1.2869999999999999},"min":{"1min":0.92000000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.514,"5min":1.72,"15min":1.72},"last":1.274},{"interface":"front","average":{"1min":1.278,"5min":1.28,"15min":1.298},"min":{"1min":1.0600000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.472,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.296}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5600,"kb_used_data":344,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366240,"statfs":{"total":96636764160,"available":96631029760,"internally_reserved":0,"allocated":352256,"data_stored":171634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.85399999999999998},"min":{"1min":0.55200000000000005,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1619999999999999,"5min":6.2939999999999996,"15min":6.2939999999999996},"last":0.627},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.84299999999999997,"15min":0.97999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.99099999999999999,"5min":6.149,"15min":6.149},"last":0.80300000000000005}]},{"osd":1,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.84999999999999998,"15min":1.2250000000000001},"min":{"1min":0.57799999999999996,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0309999999999999,"5min":14.689,"15min":14.689},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":1.014,"15min":1.379},"min":{"1min":0.629,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.081,"5min":15.281000000000001,"15min":15.281000000000001},"last":0.56399999999999995}]},{"osd":2,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81200000000000006,"15min":0.78800000000000003},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.006,"5min":1.345,"15min":1.345},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93999999999999995,"15min":0.94699999999999995},"min":{"1min":0.66200000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.2569999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.0720000000000001,"15min":1.0900000000000001},"min":{"1min":0.89600000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0389999999999999,"15min":1.097},"min":{"1min":0.70899999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3939999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.83699999999999997}]},{"osd":5,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.0960000000000001,"15min":1.163},"min":{"1min":0.89600000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.3300000000000001,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.165,"15min":1.194},"min":{"1min":0.89300000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.369,"5min":1.792,"15min":1.792},"last":0.96299999999999997}]},{"osd":6,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.21,"15min":1.21},"min":{"1min":0.85099999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.238,"15min":1.248},"min":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.776,"5min":1.863,"15min":1.863},"last":0.85399999999999998}]},{"osd":7,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2,"15min":1.228},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.956,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2450000000000001,"15min":1.2470000000000001},"min":{"1min":0.95099999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.732,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.874}]}]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5584,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366256,"statfs":{"total":96636764160,"available":96631046144,"internally_reserved":0,"allocated":335872,"data_stored":162650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.24,"5min":1.409,"15min":1.409},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1879999999999999,"5min":1.254,"15min":1.254},"last":1.155}]},{"osd":1,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.80800000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.069,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84199999999999997,"15min":0.85899999999999999},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0640000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.81899999999999995}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1699999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.97499999999999998,"15min":0.995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2110000000000001,"5min":2.016,"15min":2.016},"last":1.2210000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.089,"15min":1.103},"min":{"1min":0.74299999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.3089999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.264},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1040000000000001,"15min":1.1339999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.528,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.6719999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1160000000000001,"15min":1.119},"min":{"1min":0.81599999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.4910000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.351},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.159,"15min":1.1910000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.704,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.4159999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1739999999999999,"15min":1.214},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.571,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.321},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.27,"15min":1.292},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.6240000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.6299999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2869999999999999,"15min":1.2989999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.601,"5min":2.137,"15min":2.137},"last":1.736},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2669999999999999,"15min":1.296},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.5660000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":2.6539999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5592,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366248,"statfs":{"total":96636764160,"available":96631037952,"internally_reserved":0,"allocated":344064,"data_stored":163550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67900000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0469999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.012,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.622}]},{"osd":2,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.52000000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.1479999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.88800000000000001,"15min":0.90900000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.194,"5min":1.726,"15min":1.726},"last":0.75700000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90100000000000002,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3420000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.91100000000000003},"min":{"1min":0.64000000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2210000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.72099999999999997}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.1080000000000001,"15min":1.1499999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.411,"5min":1.893,"15min":1.893},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.127,"15min":1.133},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.466,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.256}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1140000000000001,"15min":1.1240000000000001},"min":{"1min":0.91100000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5189999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.302},{"interface":"front","average":{"1min":1.085,"5min":1.117,"15min":1.1319999999999999},"min":{"1min":0.73099999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4450000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.99199999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.143,"15min":1.167},"min":{"1min":0.84999999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.4019999999999999,"5min":1.508,"15min":1.508},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.226,"15min":1.25},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.583,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.0669999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.282,"15min":1.3109999999999999},"min":{"1min":0.98599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.669,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.2},{"interface":"front","average":{"1min":1.198,"5min":1.252,"15min":1.2929999999999999},"min":{"1min":0.98399999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.5469999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.0920000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6088,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365752,"statfs":{"total":96636764160,"available":96630530048,"internally_reserved":0,"allocated":393216,"data_stored":692192,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68100000000000005,"15min":0.70699999999999996},"min":{"1min":0.53800000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.80500000000000005,"15min":0.81699999999999995},"min":{"1min":0.55800000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.208,"5min":1.377,"15min":1.377},"last":0.98299999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.121,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.746,"15min":0.745},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.2370000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.75800000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.63,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0429999999999999,"5min":1.075,"15min":1.075},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.999,"5min":0.96999999999999997,"15min":0.98699999999999999},"min":{"1min":0.70899999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.018,"15min":1.0489999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.3879999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.101,"15min":1.1359999999999999},"min":{"1min":0.91300000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4930000000000001,"5min":1.655,"15min":1.655},"last":1.1479999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0820000000000001,"15min":1.1060000000000001},"min":{"1min":0.79600000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.476,"5min":1.482,"15min":1.482},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.1850000000000001,"15min":1.208},"min":{"1min":0.94999999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4610000000000001,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.0940000000000001}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.133,"15min":1.139},"min":{"1min":0.79900000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.226,"15min":1.26},"min":{"1min":0.98199999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.1899999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2549999999999999,"15min":1.3029999999999999},"min":{"1min":1.0429999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.5720000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.375},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.224,"15min":1.25},"min":{"1min":0.89600000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.6679999999999999,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.0629999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:07:14.116 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T01:07:14.407 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:07:14.407 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"fsid":"43b20d8a-eabf-4c8a-988f-eaf5c9521071","created":"2023-09-30T01:01:55.405921+0000","modified":"2023-09-30T01:07:09.531710+0000","last_up_change":"2023-09-30T01:02:04.607879+0000","last_in_change":"2023-09-30T01:01:57.341399+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T01:02:03.654754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T01:02:06.017647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2e24b7d6-a497-485e-b99b-0bfa85c43338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6802","nonce":31154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6804","nonce":31154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6807","nonce":31154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6805","nonce":31154}]},"public_addr":"172.21.15.37:6802/31154","cluster_addr":"172.21.15.37:6804/31154","heartbeat_back_addr":"172.21.15.37:6807/31154","heartbeat_front_addr":"172.21.15.37:6805/31154","state":["exists","up"]},{"osd":1,"uuid":"a8f0a7a5-328d-4868-a30d-22e3b5b832ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6808","nonce":31153}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6809","nonce":31153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6811","nonce":31153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6810","nonce":31153}]},"public_addr":"172.21.15.37:6808/31153","cluster_addr":"172.21.15.37:6809/31153","heartbeat_back_addr":"172.21.15.37:6811/31153","heartbeat_front_addr":"172.21.15.37:6810/31153","state":["exists","up"]},{"osd":2,"uuid":"376a9ecf-3a2c-4bd5-add5-0d1e64bece9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6812","nonce":31155}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6813","nonce":31155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6815","nonce":31155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6814","nonce":31155}]},"public_addr":"172.21.15.37:6812/31155","cluster_addr":"172.21.15.37:6813/31155","heartbeat_back_addr":"172.21.15.37:6815/31155","heartbeat_front_addr":"172.21.15.37:6814/31155","state":["exists","up"]},{"osd":3,"uuid":"4b7b474d-a9a6-4d45-b263-da4ddeb0a5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6800","nonce":31152}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6801","nonce":31152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6806","nonce":31152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.37:6803","nonce":31152}]},"public_addr":"172.21.15.37:6800/31152","cluster_addr":"172.21.15.37:6801/31152","heartbeat_back_addr":"172.21.15.37:6806/31152","heartbeat_front_addr":"172.21.15.37:6803/31152","state":["exists","up"]},{"osd":4,"uuid":"949fb796-6e09-474e-b21e-8ba7a52c1d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6804","nonce":31190}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6805","nonce":31190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6807","nonce":31190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6806","nonce":31190}]},"public_addr":"172.21.15.50:6804/31190","cluster_addr":"172.21.15.50:6805/31190","heartbeat_back_addr":"172.21.15.50:6807/31190","heartbeat_front_addr":"172.21.15.50:6806/31190","state":["exists","up"]},{"osd":5,"uuid":"869b70fb-be90-4d48-b9c6-a1f2d6e1682f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6800","nonce":31191}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6801","nonce":31191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6803","nonce":31191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6802","nonce":31191}]},"public_addr":"172.21.15.50:6800/31191","cluster_addr":"172.21.15.50:6801/31191","heartbeat_back_addr":"172.21.15.50:6803/31191","heartbeat_front_addr":"172.21.15.50:6802/31191","state":["exists","up"]},{"osd":6,"uuid":"bddff542-e807-42f4-91e1-c6b3db62d7a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6810","nonce":31192}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6812","nonce":31192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6815","nonce":31192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6813","nonce":31192}]},"public_addr":"172.21.15.50:6810/31192","cluster_addr":"172.21.15.50:6812/31192","heartbeat_back_addr":"172.21.15.50:6815/31192","heartbeat_front_addr":"172.21.15.50:6813/31192","state":["exists","up"]},{"osd":7,"uuid":"a1448ff4-9ea4-4fd3-a940-ed51cbb9b3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6808","nonce":31215}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6809","nonce":31215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6814","nonce":31215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.50:6811","nonce":31215}]},"public_addr":"172.21.15.50:6808/31215","cluster_addr":"172.21.15.50:6809/31215","heartbeat_back_addr":"172.21.15.50:6814/31215","heartbeat_front_addr":"172.21.15.50:6811/31215","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:07:08.000197+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:06:00.793465+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:06:47.324943+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:07:06.720443+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:05:56.394856+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:07:05.708097+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:07:09.033957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:06:40.351536+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T01:07:14.751 INFO:tasks.ceph.osd.2.smithi037.stderr:2023-09-30T01:07:14.746+0000 7f5d7bac2700 -1 osd.2 47 handle_osd_map injecting map crc failure 2023-09-30T01:07:15.104 INFO:tasks.ceph.osd.5.smithi050.stderr:2023-09-30T01:07:15.102+0000 7f711c629700 -1 osd.5 47 handle_osd_map injecting map crc failure 2023-09-30T01:07:15.422 INFO:tasks.ceph:Scrubbing osd.0 2023-09-30T01:07:15.423 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:07:15.577 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2023-09-30T01:07:15.578 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:07:15.578 INFO:teuthology.orchestra.run.smithi037.stdout:} 2023-09-30T01:07:15.591 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-09-30T01:07:15.892 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 0 to deep-scrub 2023-09-30T01:07:15.901 INFO:tasks.ceph:Scrubbing osd.1 2023-09-30T01:07:15.902 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:07:16.044 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2023-09-30T01:07:16.045 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:07:16.045 INFO:teuthology.orchestra.run.smithi037.stdout:} 2023-09-30T01:07:16.057 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-09-30T01:07:16.347 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 1 to deep-scrub 2023-09-30T01:07:16.360 INFO:tasks.ceph:Scrubbing osd.2 2023-09-30T01:07:16.360 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:07:16.507 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2023-09-30T01:07:16.508 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:07:16.508 INFO:teuthology.orchestra.run.smithi037.stdout:} 2023-09-30T01:07:16.520 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-09-30T01:07:16.816 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 2 to deep-scrub 2023-09-30T01:07:16.826 INFO:tasks.ceph:Scrubbing osd.3 2023-09-30T01:07:16.826 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:07:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2023-09-30T01:07:16.976 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:07:16.976 INFO:teuthology.orchestra.run.smithi037.stdout:} 2023-09-30T01:07:16.989 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-09-30T01:07:17.292 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 3 to deep-scrub 2023-09-30T01:07:17.302 INFO:tasks.ceph:Scrubbing osd.4 2023-09-30T01:07:17.302 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:07:17.443 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2023-09-30T01:07:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:07:17.443 INFO:teuthology.orchestra.run.smithi037.stdout:} 2023-09-30T01:07:17.455 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-09-30T01:07:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 4 to deep-scrub 2023-09-30T01:07:17.759 INFO:tasks.ceph:Scrubbing osd.5 2023-09-30T01:07:17.760 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:07:17.913 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2023-09-30T01:07:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:07:17.913 INFO:teuthology.orchestra.run.smithi037.stdout:} 2023-09-30T01:07:17.926 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-09-30T01:07:18.220 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 5 to deep-scrub 2023-09-30T01:07:18.236 INFO:tasks.ceph:Scrubbing osd.6 2023-09-30T01:07:18.236 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:07:18.383 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2023-09-30T01:07:18.383 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:07:18.383 INFO:teuthology.orchestra.run.smithi037.stdout:} 2023-09-30T01:07:18.397 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-09-30T01:07:18.686 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 6 to deep-scrub 2023-09-30T01:07:18.700 INFO:tasks.ceph:Scrubbing osd.7 2023-09-30T01:07:18.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:07:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2023-09-30T01:07:18.849 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:07:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:} 2023-09-30T01:07:18.861 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-09-30T01:07:19.155 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 7 to deep-scrub 2023-09-30T01:07:19.165 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:07:19.460 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:07:19.463 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2023-09-30T01:07:19.475 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2023-09-30T01:07:17.683814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":45876,"kb_used_data":2868,"kb_used_omap":0,"kb_used_meta":42944,"kb_avail":754928844,"statfs":{"total":773094113280,"available":773047136256,"internally_reserved":0,"allocated":2936832,"data_stored":2428901,"data_compressed":11862,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":43974656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.138600"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-09-30T01:07:10.006504+0000","last_change":"2023-09-30T01:06:20.261279+0000","last_active":"2023-09-30T01:07:10.006504+0000","last_peered":"2023-09-30T01:07:10.006504+0000","last_clean":"2023-09-30T01:07:10.006504+0000","last_became_active":"2023-09-30T01:02:07.769178+0000","last_became_peered":"2023-09-30T01:02:07.769178+0000","last_unstale":"2023-09-30T01:07:10.006504+0000","last_undegraded":"2023-09-30T01:07:10.006504+0000","last_fullsized":"2023-09-30T01:07:10.006504+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:20.261102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:20.261102+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:36.156659+0000","scrub_duration":0.20364980799999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.638562+0000","last_change":"2023-09-30T01:06:18.345145+0000","last_active":"2023-09-30T01:07:08.638562+0000","last_peered":"2023-09-30T01:07:08.638562+0000","last_clean":"2023-09-30T01:07:08.638562+0000","last_became_active":"2023-09-30T01:02:07.872154+0000","last_became_peered":"2023-09-30T01:02:07.872154+0000","last_unstale":"2023-09-30T01:07:08.638562+0000","last_undegraded":"2023-09-30T01:07:08.638562+0000","last_fullsized":"2023-09-30T01:07:08.638562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:18.344893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:06:18.344893+0000","last_clean_scrub_stamp":"2023-09-30T01:06:18.344893+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:22.660473+0000","scrub_duration":0.20445159399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.548345+0000","last_change":"2023-09-30T01:06:04.600417+0000","last_active":"2023-09-30T01:07:09.548345+0000","last_peered":"2023-09-30T01:07:09.548345+0000","last_clean":"2023-09-30T01:07:09.548345+0000","last_became_active":"2023-09-30T01:02:07.873496+0000","last_became_peered":"2023-09-30T01:02:07.873496+0000","last_unstale":"2023-09-30T01:07:09.548345+0000","last_undegraded":"2023-09-30T01:07:09.548345+0000","last_fullsized":"2023-09-30T01:07:09.548345+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:04.600148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:03:35.046769+0000","last_clean_scrub_stamp":"2023-09-30T01:06:04.600148+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:18.895144+0000","scrub_duration":0.30469532599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":77,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.538394+0000","last_change":"2023-09-30T01:06:43.248656+0000","last_active":"2023-09-30T01:07:08.538394+0000","last_peered":"2023-09-30T01:07:08.538394+0000","last_clean":"2023-09-30T01:07:08.538394+0000","last_became_active":"2023-09-30T01:02:07.872317+0000","last_became_peered":"2023-09-30T01:02:07.872317+0000","last_unstale":"2023-09-30T01:07:08.538394+0000","last_undegraded":"2023-09-30T01:07:08.538394+0000","last_fullsized":"2023-09-30T01:07:08.538394+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:43.248242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:43.248242+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:07.553083+0000","scrub_duration":0.103841286,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.538172+0000","last_change":"2023-09-30T01:06:07.319639+0000","last_active":"2023-09-30T01:07:08.538172+0000","last_peered":"2023-09-30T01:07:08.538172+0000","last_clean":"2023-09-30T01:07:08.538172+0000","last_became_active":"2023-09-30T01:02:07.677592+0000","last_became_peered":"2023-09-30T01:02:07.677592+0000","last_unstale":"2023-09-30T01:07:08.538172+0000","last_undegraded":"2023-09-30T01:07:08.538172+0000","last_fullsized":"2023-09-30T01:07:08.538172+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:07.319360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:03:15.943036+0000","last_clean_scrub_stamp":"2023-09-30T01:06:07.319360+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:26.661423+0000","scrub_duration":0.304743127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.537523+0000","last_change":"2023-09-30T01:06:06.052328+0000","last_active":"2023-09-30T01:07:08.537523+0000","last_peered":"2023-09-30T01:07:08.537523+0000","last_clean":"2023-09-30T01:07:08.537523+0000","last_became_active":"2023-09-30T01:02:07.775147+0000","last_became_peered":"2023-09-30T01:02:07.775147+0000","last_unstale":"2023-09-30T01:07:08.537523+0000","last_undegraded":"2023-09-30T01:07:08.537523+0000","last_fullsized":"2023-09-30T01:07:08.537523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:06.052079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:06.052079+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:36.034258+0000","scrub_duration":0.203916498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.548211+0000","last_change":"2023-09-30T01:06:58.156271+0000","last_active":"2023-09-30T01:07:09.548211+0000","last_peered":"2023-09-30T01:07:09.548211+0000","last_clean":"2023-09-30T01:07:09.548211+0000","last_became_active":"2023-09-30T01:02:07.772379+0000","last_became_peered":"2023-09-30T01:02:07.772379+0000","last_unstale":"2023-09-30T01:07:09.548211+0000","last_undegraded":"2023-09-30T01:07:09.548211+0000","last_fullsized":"2023-09-30T01:07:09.548211+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:58.156016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:06:58.156016+0000","last_clean_scrub_stamp":"2023-09-30T01:06:58.156016+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:26.133597+0000","scrub_duration":0.60520065199999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-09-30T01:07:08.538151+0000","last_change":"2023-09-30T01:06:15.101701+0000","last_active":"2023-09-30T01:07:08.538151+0000","last_peered":"2023-09-30T01:07:08.538151+0000","last_clean":"2023-09-30T01:07:08.538151+0000","last_became_active":"2023-09-30T01:02:07.773993+0000","last_became_peered":"2023-09-30T01:02:07.773993+0000","last_unstale":"2023-09-30T01:07:08.538151+0000","last_undegraded":"2023-09-30T01:07:08.538151+0000","last_fullsized":"2023-09-30T01:07:08.538151+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:06:15.101414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:06.639364+0000","last_clean_scrub_stamp":"2023-09-30T01:06:15.101414+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:40.776761+0000","scrub_duration":0.20410309200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":221,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-09-30T01:07:09.548321+0000","last_change":"2023-09-30T01:06:10.629812+0000","last_active":"2023-09-30T01:07:09.548321+0000","last_peered":"2023-09-30T01:07:09.548321+0000","last_clean":"2023-09-30T01:07:09.548321+0000","last_became_active":"2023-09-30T01:02:06.336950+0000","last_became_peered":"2023-09-30T01:02:06.336950+0000","last_unstale":"2023-09-30T01:07:09.548321+0000","last_undegraded":"2023-09-30T01:07:09.548321+0000","last_fullsized":"2023-09-30T01:07:09.548321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'117","last_scrub_stamp":"2023-09-30T01:06:10.629534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:02:04.607879+0000","last_clean_scrub_stamp":"2023-09-30T01:06:10.629534+0000","objects_scrubbed":2,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:07:21.216419+0000","scrub_duration":0.30592072999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11862,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542209,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6168,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94365672,"statfs":{"total":96636764160,"available":96630448128,"internally_reserved":0,"allocated":409600,"data_stored":702546,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90500000000000003,"15min":0.91500000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.73099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.306,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.77900000000000003}]},{"osd":1,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.5700000000000001,"15min":1.0880000000000001},"min":{"1min":0.65400000000000003,"5min":0.62,"15min":0.62},"max":{"1min":1.3640000000000001,"5min":78.195999999999998,"15min":78.195999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.98599999999999999,"15min":1.0429999999999999},"min":{"1min":0.70499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2330000000000001,"5min":1.452,"15min":1.452},"last":1.0820000000000001}]},{"osd":2,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.92700000000000005},"min":{"1min":0.67700000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.0449999999999999}]},{"osd":3,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.99199999999999999,"15min":1.0169999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2350000000000001,"5min":1.363,"15min":1.363},"last":1.216},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0529999999999999,"15min":1.0489999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2869999999999999,"5min":1.425,"15min":1.425},"last":1.171}]},{"osd":4,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.98299999999999998,"15min":1.0089999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.105,"15min":1.157},"min":{"1min":0.70799999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3280000000000001,"5min":1.486,"15min":1.486},"last":1.3200000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0960000000000001,"15min":1.081},"min":{"1min":0.89000000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.389,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.268},{"interface":"front","average":{"1min":1.198,"5min":1.1830000000000001,"15min":1.1990000000000001},"min":{"1min":0.78300000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.242}]},{"osd":6,"last update":"Sat Sep 30 01:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2529999999999999,"15min":1.2749999999999999},"min":{"1min":0.98499999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.474,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2629999999999999,"15min":1.2769999999999999},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.573,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.385}]}]},{"osd":6,"up_from":14,"seq":60129542209,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5592,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366248,"statfs":{"total":96636764160,"available":96631037952,"internally_reserved":0,"allocated":344064,"data_stored":167526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92000000000000004,"15min":0.93200000000000005},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.29,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.83099999999999996}]},{"osd":1,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.96399999999999997,"15min":0.95999999999999996},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93100000000000005,"15min":0.92600000000000005},"min":{"1min":0.68700000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.327,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.2609999999999999}]},{"osd":2,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.70199999999999996,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.353,"5min":1.462,"15min":1.462},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.482,"15min":1.24},"min":{"1min":0.65400000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.2130000000000001,"5min":47.942,"15min":47.942},"last":1.984}]},{"osd":3,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.038,"15min":1.0940000000000001},"min":{"1min":0.67900000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.304,"5min":1.345,"15min":1.345},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.2569999999999999,"15min":1.107},"min":{"1min":0.69199999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.373,"5min":20.649999999999999,"15min":20.649999999999999},"last":1.0209999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.98399999999999999,"15min":0.99199999999999999},"min":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.327,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.355},{"interface":"front","average":{"1min":1.054,"5min":1.0369999999999999,"15min":1.046},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.298,"5min":1.405,"15min":1.405},"last":1.8440000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.113,"15min":1.113},"min":{"1min":0.83399999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.3420000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.107},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.206,"15min":1.2569999999999999},"min":{"1min":0.93500000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.577,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":2.028}]},{"osd":7,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1240000000000001,"15min":1.1579999999999999},"min":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3029999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.232,"15min":1.246},"min":{"1min":0.76600000000000001,"5min":0.752,"15min":0.752},"max":{"1min":1.429,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.681}]}]},{"osd":4,"up_from":14,"seq":60129542210,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5612,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366228,"statfs":{"total":96636764160,"available":96631017472,"internally_reserved":0,"allocated":364544,"data_stored":174190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.90800000000000003,"15min":0.91000000000000003},"min":{"1min":0.55100000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.504,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.504},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.95099999999999996,"15min":0.91100000000000003},"min":{"1min":0.69399999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.339,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.87}]},{"osd":1,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.96499999999999997,"15min":0.96699999999999997},"min":{"1min":0.74199999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5149999999999999,"5min":1.655,"15min":1.655},"last":1.4450000000000001},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.91900000000000004},"min":{"1min":0.70499999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.222,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.72999999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.96599999999999997,"15min":0.97999999999999998},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1,"15min":1.038},"min":{"1min":0.747,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3580000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.3580000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0189999999999999,"15min":1.038},"min":{"1min":0.77600000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.0880000000000001,"15min":1.0800000000000001},"min":{"1min":0.79000000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.744}]},{"osd":5,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.95999999999999996,"15min":0.93400000000000005},"min":{"1min":0.63400000000000001,"5min":0.624,"15min":0.624},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2170000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.615,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.5629999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.079,"15min":1.0940000000000001},"min":{"1min":0.82399999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.143,"15min":1.145},"min":{"1min":0.79700000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.849}]},{"osd":7,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.222,"15min":1.25},"min":{"1min":0.74099999999999999,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":2.343},{"interface":"front","average":{"1min":1.339,"5min":1.294,"15min":1.282},"min":{"1min":1.125,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.99}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5592,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366248,"statfs":{"total":96636764160,"available":96631037952,"internally_reserved":0,"allocated":344064,"data_stored":168645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.93100000000000005,"15min":0.91100000000000003},"min":{"1min":0.67900000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.228,"5min":1.288,"15min":1.288},"last":0.998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.91800000000000004,"15min":0.89200000000000002},"min":{"1min":0.60799999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2470000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.038}]},{"osd":1,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.94599999999999995,"15min":0.97599999999999998},"min":{"1min":0.61799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3280000000000001,"5min":1.387,"15min":1.387},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.995,"5min":0.97699999999999998,"15min":0.96199999999999997},"min":{"1min":0.70699999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2889999999999999,"5min":1.454,"15min":1.454},"last":0.60799999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93799999999999994,"15min":0.89800000000000002},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1890000000000001,"5min":1.448,"15min":1.448},"last":1.085},{"interface":"front","average":{"1min":0.996,"5min":1.042,"15min":1.0760000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.264,"5min":1.349,"15min":1.349},"last":1.139}]},{"osd":3,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.052,"15min":1.0720000000000001},"min":{"1min":0.70999999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.448,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.071,"5min":1.0549999999999999,"15min":1.0529999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.3720000000000001,"5min":1.466,"15min":1.466},"last":1.2290000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96899999999999997,"15min":1.0069999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.133,"5min":1.1100000000000001,"15min":1.119},"min":{"1min":0.71999999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.419,"5min":1.454,"15min":1.454},"last":1.325}]},{"osd":6,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.129,"15min":1.153},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3779999999999999,"5min":2.097,"15min":2.097},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.1459999999999999,"15min":1.163},"min":{"1min":0.90200000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.353,"5min":1.55,"15min":1.55},"last":1.341}]},{"osd":7,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2490000000000001,"15min":1.2869999999999999},"min":{"1min":0.92000000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.514,"5min":1.72,"15min":1.72},"last":1.274},{"interface":"front","average":{"1min":1.278,"5min":1.28,"15min":1.298},"min":{"1min":1.0600000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.472,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.296}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5600,"kb_used_data":344,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366240,"statfs":{"total":96636764160,"available":96631029760,"internally_reserved":0,"allocated":352256,"data_stored":171634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.85399999999999998},"min":{"1min":0.55200000000000005,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1619999999999999,"5min":6.2939999999999996,"15min":6.2939999999999996},"last":0.627},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.84299999999999997,"15min":0.97999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.99099999999999999,"5min":6.149,"15min":6.149},"last":0.80300000000000005}]},{"osd":1,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.84999999999999998,"15min":1.2250000000000001},"min":{"1min":0.57799999999999996,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0309999999999999,"5min":14.689,"15min":14.689},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":1.014,"15min":1.379},"min":{"1min":0.629,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.081,"5min":15.281000000000001,"15min":15.281000000000001},"last":0.56399999999999995}]},{"osd":2,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81200000000000006,"15min":0.78800000000000003},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.006,"5min":1.345,"15min":1.345},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93999999999999995,"15min":0.94699999999999995},"min":{"1min":0.66200000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.2569999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.0720000000000001,"15min":1.0900000000000001},"min":{"1min":0.89600000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0389999999999999,"15min":1.097},"min":{"1min":0.70899999999999996,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3939999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.83699999999999997}]},{"osd":5,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.0960000000000001,"15min":1.163},"min":{"1min":0.89600000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.3300000000000001,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.165,"15min":1.194},"min":{"1min":0.89300000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.369,"5min":1.792,"15min":1.792},"last":0.96299999999999997}]},{"osd":6,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.21,"15min":1.21},"min":{"1min":0.85099999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.238,"15min":1.248},"min":{"1min":0.99199999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.776,"5min":1.863,"15min":1.863},"last":0.85399999999999998}]},{"osd":7,"last update":"Sat Sep 30 01:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2,"15min":1.228},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.956,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2450000000000001,"15min":1.2470000000000001},"min":{"1min":0.95099999999999996,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.732,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.874}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5600,"kb_used_data":344,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366240,"statfs":{"total":96636764160,"available":96631029760,"internally_reserved":0,"allocated":352256,"data_stored":171306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.24,"5min":1.409,"15min":1.409},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78400000000000003,"15min":0.81000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1879999999999999,"5min":1.254,"15min":1.254},"last":0.81299999999999994}]},{"osd":1,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.80800000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.069,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84199999999999997,"15min":0.85899999999999999},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0640000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.66600000000000004}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1699999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.97499999999999998,"15min":0.995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2110000000000001,"5min":2.016,"15min":2.016},"last":1.0089999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.089,"15min":1.103},"min":{"1min":0.74299999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.3089999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1040000000000001,"15min":1.1339999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.528,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.0389999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1160000000000001,"15min":1.119},"min":{"1min":0.81599999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.4910000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.081},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.159,"15min":1.1910000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.704,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.248}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1739999999999999,"15min":1.214},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.571,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.27,"15min":1.292},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.6240000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.107}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2869999999999999,"15min":1.2989999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.601,"5min":2.137,"15min":2.137},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2669999999999999,"15min":1.296},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.5660000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.137}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5608,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366232,"statfs":{"total":96636764160,"available":96631021568,"internally_reserved":0,"allocated":360448,"data_stored":172206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67900000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0469999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.012,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.68000000000000005}]},{"osd":2,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.52000000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.1479999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.88800000000000001,"15min":0.90900000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.194,"5min":1.726,"15min":1.726},"last":0.92300000000000004}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90100000000000002,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3420000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.016},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.91100000000000003},"min":{"1min":0.64000000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2210000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.79300000000000004}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.1080000000000001,"15min":1.1499999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.411,"5min":1.893,"15min":1.893},"last":1.135},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.127,"15min":1.133},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.466,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":0.96699999999999997}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1140000000000001,"15min":1.1240000000000001},"min":{"1min":0.91100000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5189999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.071},{"interface":"front","average":{"1min":1.085,"5min":1.117,"15min":1.1319999999999999},"min":{"1min":0.73099999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4450000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.036}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.143,"15min":1.167},"min":{"1min":0.84999999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.4019999999999999,"5min":1.508,"15min":1.508},"last":1.111},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.226,"15min":1.25},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.583,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.3759999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.282,"15min":1.3109999999999999},"min":{"1min":0.98599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.669,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.173},{"interface":"front","average":{"1min":1.198,"5min":1.252,"15min":1.2929999999999999},"min":{"1min":0.98399999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.5469999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.268}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6104,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365736,"statfs":{"total":96636764160,"available":96630513664,"internally_reserved":0,"allocated":409600,"data_stored":700848,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68100000000000005,"15min":0.70699999999999996},"min":{"1min":0.53800000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.80500000000000005,"15min":0.81699999999999995},"min":{"1min":0.55800000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.208,"5min":1.377,"15min":1.377},"last":1.083}]},{"osd":2,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.64000000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.121,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.746,"15min":0.745},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.2370000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.87}]},{"osd":3,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.63,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0429999999999999,"5min":1.075,"15min":1.075},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.999,"5min":0.96999999999999997,"15min":0.98699999999999999},"min":{"1min":0.70899999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.173}]},{"osd":4,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.018,"15min":1.0489999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.3879999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.012},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.101,"15min":1.1359999999999999},"min":{"1min":0.91300000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4930000000000001,"5min":1.655,"15min":1.655},"last":1.3100000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0820000000000001,"15min":1.1060000000000001},"min":{"1min":0.79600000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.476,"5min":1.482,"15min":1.482},"last":1.113},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.1850000000000001,"15min":1.208},"min":{"1min":0.94999999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4610000000000001,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.21}]},{"osd":6,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.133,"15min":1.139},"min":{"1min":0.79900000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.05},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.226,"15min":1.26},"min":{"1min":0.98199999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.137}]},{"osd":7,"last update":"Sat Sep 30 01:06:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2549999999999999,"15min":1.3029999999999999},"min":{"1min":1.0429999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.5720000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.224,"15min":1.25},"min":{"1min":0.89600000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.6679999999999999,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.268}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:07:19.479 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-09-30T01:06:20.261102+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=6, tm_sec=20, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=7, tm_sec=14, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:07:19.480 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-30T01:06:18.344893+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=6, tm_sec=18, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=7, tm_sec=14, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:07:19.480 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-09-30T01:06:04.600148+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=6, tm_sec=4, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=7, tm_sec=14, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:07:19.480 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-30T01:06:43.248242+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=6, tm_sec=43, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=7, tm_sec=14, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:07:19.480 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-09-30T01:06:07.319360+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=6, tm_sec=7, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=7, tm_sec=14, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:07:19.480 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-09-30T01:06:06.052079+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=6, tm_sec=6, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=7, tm_sec=14, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:07:19.480 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-09-30T01:06:58.156016+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=6, tm_sec=58, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=7, tm_sec=14, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:07:19.480 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-30T01:06:15.101414+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=6, tm_sec=15, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=7, tm_sec=14, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:07:19.480 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-30T01:06:10.629534+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=6, tm_sec=10, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=7, tm_sec=14, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:07:19.481 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-30T01:07:25.345 INFO:tasks.ceph.osd.4.smithi050.stderr:2023-09-30T01:07:25.342+0000 7f8652074700 -1 osd.4 48 handle_osd_map injecting map crc failure 2023-09-30T01:07:39.481 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:07:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: 2023-09-30T01:07:39.795 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2023-09-30T01:07:39.807 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2023-09-30T01:07:39.690374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":46084,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":43072,"kb_avail":754928636,"statfs":{"total":773094113280,"available":773046923264,"internally_reserved":0,"allocated":3018752,"data_stored":2487205,"data_compressed":11862,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":44105728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003592"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":81,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-09-30T01:07:24.136346+0000","last_change":"2023-09-30T01:07:24.136222+0000","last_active":"2023-09-30T01:07:24.136346+0000","last_peered":"2023-09-30T01:07:24.136346+0000","last_clean":"2023-09-30T01:07:24.136346+0000","last_became_active":"2023-09-30T01:02:07.769178+0000","last_became_peered":"2023-09-30T01:02:07.769178+0000","last_unstale":"2023-09-30T01:07:24.136346+0000","last_undegraded":"2023-09-30T01:07:24.136346+0000","last_fullsized":"2023-09-30T01:07:24.136346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:07:24.135961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:07:24.135961+0000","last_clean_scrub_stamp":"2023-09-30T01:07:24.135961+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:25.636469+0000","scrub_duration":0.40443495899999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-09-30T01:07:29.361528+0000","last_change":"2023-09-30T01:07:29.361459+0000","last_active":"2023-09-30T01:07:29.361528+0000","last_peered":"2023-09-30T01:07:29.361528+0000","last_clean":"2023-09-30T01:07:29.361528+0000","last_became_active":"2023-09-30T01:02:07.872154+0000","last_became_peered":"2023-09-30T01:02:07.872154+0000","last_unstale":"2023-09-30T01:07:29.361528+0000","last_undegraded":"2023-09-30T01:07:29.361528+0000","last_fullsized":"2023-09-30T01:07:29.361528+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:07:29.361184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:07:29.361184+0000","last_clean_scrub_stamp":"2023-09-30T01:07:29.361184+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:46.167183+0000","scrub_duration":0.20428734100000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":81,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-09-30T01:07:20.873196+0000","last_change":"2023-09-30T01:07:20.873017+0000","last_active":"2023-09-30T01:07:20.873196+0000","last_peered":"2023-09-30T01:07:20.873196+0000","last_clean":"2023-09-30T01:07:20.873196+0000","last_became_active":"2023-09-30T01:02:07.873496+0000","last_became_peered":"2023-09-30T01:02:07.873496+0000","last_unstale":"2023-09-30T01:07:20.873196+0000","last_undegraded":"2023-09-30T01:07:20.873196+0000","last_fullsized":"2023-09-30T01:07:20.873196+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:07:20.872767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:07:20.872767+0000","last_clean_scrub_stamp":"2023-09-30T01:07:20.872767+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:22.074749+0000","scrub_duration":0.30450431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-09-30T01:07:22.361889+0000","last_change":"2023-09-30T01:07:22.361733+0000","last_active":"2023-09-30T01:07:22.361889+0000","last_peered":"2023-09-30T01:07:22.361889+0000","last_clean":"2023-09-30T01:07:22.361889+0000","last_became_active":"2023-09-30T01:02:07.872317+0000","last_became_peered":"2023-09-30T01:02:07.872317+0000","last_unstale":"2023-09-30T01:07:22.361889+0000","last_undegraded":"2023-09-30T01:07:22.361889+0000","last_fullsized":"2023-09-30T01:07:22.361889+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:07:22.361322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:07:22.361322+0000","last_clean_scrub_stamp":"2023-09-30T01:07:22.361322+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:42.794677+0000","scrub_duration":0.30448957700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-09-30T01:07:32.748643+0000","last_change":"2023-09-30T01:07:32.748575+0000","last_active":"2023-09-30T01:07:32.748643+0000","last_peered":"2023-09-30T01:07:32.748643+0000","last_clean":"2023-09-30T01:07:32.748643+0000","last_became_active":"2023-09-30T01:02:07.677592+0000","last_became_peered":"2023-09-30T01:02:07.677592+0000","last_unstale":"2023-09-30T01:07:32.748643+0000","last_undegraded":"2023-09-30T01:07:32.748643+0000","last_fullsized":"2023-09-30T01:07:32.748643+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:07:32.748295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:07:32.748295+0000","last_clean_scrub_stamp":"2023-09-30T01:07:32.748295+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:55.365198+0000","scrub_duration":0.104626129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":81,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-09-30T01:07:22.230635+0000","last_change":"2023-09-30T01:07:22.230487+0000","last_active":"2023-09-30T01:07:22.230635+0000","last_peered":"2023-09-30T01:07:22.230635+0000","last_clean":"2023-09-30T01:07:22.230635+0000","last_became_active":"2023-09-30T01:02:07.775147+0000","last_became_peered":"2023-09-30T01:02:07.775147+0000","last_unstale":"2023-09-30T01:07:22.230635+0000","last_undegraded":"2023-09-30T01:07:22.230635+0000","last_fullsized":"2023-09-30T01:07:22.230635+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:07:22.230092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:07:22.230092+0000","last_clean_scrub_stamp":"2023-09-30T01:07:22.230092+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:39.361589+0000","scrub_duration":0.30414824699999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-09-30T01:07:22.722891+0000","last_change":"2023-09-30T01:07:22.722780+0000","last_active":"2023-09-30T01:07:22.722891+0000","last_peered":"2023-09-30T01:07:22.722891+0000","last_clean":"2023-09-30T01:07:22.722891+0000","last_became_active":"2023-09-30T01:02:07.772379+0000","last_became_peered":"2023-09-30T01:02:07.772379+0000","last_unstale":"2023-09-30T01:07:22.722891+0000","last_undegraded":"2023-09-30T01:07:22.722891+0000","last_fullsized":"2023-09-30T01:07:22.722891+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:07:22.722532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:07:22.722532+0000","last_clean_scrub_stamp":"2023-09-30T01:07:22.722532+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:38.325035+0000","scrub_duration":0.104296468,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-09-30T01:07:34.736640+0000","last_change":"2023-09-30T01:07:34.736528+0000","last_active":"2023-09-30T01:07:34.736640+0000","last_peered":"2023-09-30T01:07:34.736640+0000","last_clean":"2023-09-30T01:07:34.736640+0000","last_became_active":"2023-09-30T01:02:07.773993+0000","last_became_peered":"2023-09-30T01:02:07.773993+0000","last_unstale":"2023-09-30T01:07:34.736640+0000","last_undegraded":"2023-09-30T01:07:34.736640+0000","last_fullsized":"2023-09-30T01:07:34.736640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:07:34.736271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:07:34.736271+0000","last_clean_scrub_stamp":"2023-09-30T01:07:34.736271+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:37.973845+0000","scrub_duration":0.0041317840000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"48'119","reported_seq":235,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-09-30T01:07:20.141557+0000","last_change":"2023-09-30T01:07:20.141444+0000","last_active":"2023-09-30T01:07:20.141557+0000","last_peered":"2023-09-30T01:07:20.141557+0000","last_clean":"2023-09-30T01:07:20.141557+0000","last_became_active":"2023-09-30T01:02:06.336950+0000","last_became_peered":"2023-09-30T01:02:06.336950+0000","last_unstale":"2023-09-30T01:07:20.141557+0000","last_undegraded":"2023-09-30T01:07:20.141557+0000","last_fullsized":"2023-09-30T01:07:20.141557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"48'119","last_scrub_stamp":"2023-09-30T01:07:20.141209+0000","last_deep_scrub":"48'119","last_deep_scrub_stamp":"2023-09-30T01:07:20.141209+0000","last_clean_scrub_stamp":"2023-09-30T01:07:20.141209+0000","objects_scrubbed":2,"log_size":119,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:08:38.519448+0000","scrub_duration":0.61334588800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11862,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542213,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6172,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94365668,"statfs":{"total":96636764160,"available":96630444032,"internally_reserved":0,"allocated":413696,"data_stored":706588,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.89700000000000002,"15min":0.91200000000000003},"min":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0649999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.92300000000000004,"15min":0.93899999999999995},"min":{"1min":0.63700000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2729999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.012}]},{"osd":1,"last update":"Sat Sep 30 01:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.571,"15min":1.089},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1379999999999999,"5min":78.195999999999998,"15min":78.195999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.008,"5min":0.97299999999999998,"15min":1.0389999999999999},"min":{"1min":0.64900000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.323,"5min":1.452,"15min":1.452},"last":1.0649999999999999}]},{"osd":2,"last update":"Sat Sep 30 01:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.95899999999999996,"15min":0.93200000000000005},"min":{"1min":0.755,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.27,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.034},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.96599999999999997,"15min":0.93899999999999995},"min":{"1min":0.68000000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.292,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.1160000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.98099999999999998,"15min":1.0129999999999999},"min":{"1min":0.69399999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.216,"5min":1.363,"15min":1.363},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0509999999999999,"15min":1.0489999999999999},"min":{"1min":0.88700000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.262,"5min":1.425,"15min":1.425},"last":0.97599999999999998}]},{"osd":4,"last update":"Sat Sep 30 01:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.95999999999999996,"15min":1.002},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3120000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0840000000000001,"15min":1.1499999999999999},"min":{"1min":0.78400000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3200000000000001,"5min":1.486,"15min":1.486},"last":0.747}]},{"osd":5,"last update":"Sat Sep 30 01:07:17 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0820000000000001,"15min":1.0760000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.268,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1699999999999999,"15min":1.194},"min":{"1min":0.80600000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.3799999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.80700000000000005}]},{"osd":6,"last update":"Sat Sep 30 01:07:17 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.228,"15min":1.2669999999999999},"min":{"1min":0.88,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.4379999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.214,"5min":1.2490000000000001,"15min":1.272},"min":{"1min":1.006,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.4410000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.1479999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542213,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5604,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366236,"statfs":{"total":96636764160,"available":96631025664,"internally_reserved":0,"allocated":356352,"data_stored":175896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92000000000000004,"15min":0.93200000000000005},"min":{"1min":0.55100000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.028},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.29,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.1659999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.96399999999999997,"15min":0.95999999999999996},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.25},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93100000000000005,"15min":0.92600000000000005},"min":{"1min":0.68700000000000006,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.327,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.82099999999999995}]},{"osd":2,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.70199999999999996,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.353,"5min":1.462,"15min":1.462},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.482,"15min":1.24},"min":{"1min":0.65400000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.2130000000000001,"5min":47.942,"15min":47.942},"last":1.119}]},{"osd":3,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.038,"15min":1.0940000000000001},"min":{"1min":0.67900000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.304,"5min":1.345,"15min":1.345},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.075,"5min":1.2569999999999999,"15min":1.107},"min":{"1min":0.69199999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.373,"5min":20.649999999999999,"15min":20.649999999999999},"last":1.0660000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.98399999999999999,"15min":0.99199999999999999},"min":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.327,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.054,"5min":1.0369999999999999,"15min":1.046},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.298,"5min":1.405,"15min":1.405},"last":0.85999999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.113,"15min":1.113},"min":{"1min":0.83399999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.3420000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.351},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.206,"15min":1.2569999999999999},"min":{"1min":0.93500000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.577,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Sat Sep 30 01:07:10 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1240000000000001,"15min":1.1579999999999999},"min":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.3029999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.232,"15min":1.246},"min":{"1min":0.76600000000000001,"5min":0.752,"15min":0.752},"max":{"1min":1.429,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.2070000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542214,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5616,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366224,"statfs":{"total":96636764160,"available":96631013376,"internally_reserved":0,"allocated":368640,"data_stored":178232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.90800000000000003,"15min":0.91000000000000003},"min":{"1min":0.55100000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.504,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.95099999999999996,"15min":0.91100000000000003},"min":{"1min":0.69399999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.339,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.73399999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.96499999999999997,"15min":0.96699999999999997},"min":{"1min":0.74199999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5149999999999999,"5min":1.655,"15min":1.655},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.91900000000000004},"min":{"1min":0.70499999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.222,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.996}]},{"osd":2,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.96599999999999997,"15min":0.97999999999999998},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":1.3},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1,"15min":1.038},"min":{"1min":0.747,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3580000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0189999999999999,"15min":1.038},"min":{"1min":0.77600000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.137,"5min":1.0880000000000001,"15min":1.0800000000000001},"min":{"1min":0.79000000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.0860000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.95999999999999996,"15min":0.93400000000000005},"min":{"1min":0.63400000000000001,"5min":0.624,"15min":0.624},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2170000000000001,"15min":1.2110000000000001},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.615,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0580000000000001}]},{"osd":6,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.079,"15min":1.0940000000000001},"min":{"1min":0.82399999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.143,"15min":1.145},"min":{"1min":0.79700000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.878}]},{"osd":7,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.222,"15min":1.25},"min":{"1min":0.74099999999999999,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.147},{"interface":"front","average":{"1min":1.339,"5min":1.294,"15min":1.282},"min":{"1min":1.125,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.254}]}]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5600,"kb_used_data":344,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366240,"statfs":{"total":96636764160,"available":96631029760,"internally_reserved":0,"allocated":352256,"data_stored":176729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.93100000000000005,"15min":0.91100000000000003},"min":{"1min":0.67900000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.228,"5min":1.288,"15min":1.288},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.91800000000000004,"15min":0.89200000000000002},"min":{"1min":0.60799999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2470000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.72299999999999998}]},{"osd":1,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.94599999999999995,"15min":0.97599999999999998},"min":{"1min":0.61799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3280000000000001,"5min":1.387,"15min":1.387},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.995,"5min":0.97699999999999998,"15min":0.96199999999999997},"min":{"1min":0.70699999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2889999999999999,"5min":1.454,"15min":1.454},"last":0.91500000000000004}]},{"osd":2,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93799999999999994,"15min":0.89800000000000002},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.1890000000000001,"5min":1.448,"15min":1.448},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.996,"5min":1.042,"15min":1.0760000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.264,"5min":1.349,"15min":1.349},"last":1.1639999999999999}]},{"osd":3,"last update":"Sat Sep 30 01:07:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.052,"15min":1.0720000000000001},"min":{"1min":0.70999999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.448,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.0549999999999999,"15min":1.0529999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.3720000000000001,"5min":1.466,"15min":1.466},"last":0.97499999999999998}]},{"osd":4,"last update":"Sat Sep 30 01:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95599999999999996,"15min":1.0029999999999999},"min":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.2789999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0920000000000001,"15min":1.113},"min":{"1min":0.66700000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.325,"5min":1.454,"15min":1.454},"last":1.282}]},{"osd":6,"last update":"Sat Sep 30 01:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.1299999999999999,"15min":1.153},"min":{"1min":0.70299999999999996,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.516,"5min":2.097,"15min":2.097},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.1439999999999999,"15min":1.1619999999999999},"min":{"1min":0.91000000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.4450000000000001,"5min":1.55,"15min":1.55},"last":1.25}]},{"osd":7,"last update":"Sat Sep 30 01:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.232,"15min":1.282},"min":{"1min":0.879,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.3320000000000001,"5min":1.72,"15min":1.72},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2829999999999999,"15min":1.2989999999999999},"min":{"1min":1.0509999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.663,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.3740000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5612,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366228,"statfs":{"total":96636764160,"available":96631017472,"internally_reserved":0,"allocated":364544,"data_stored":180004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.85399999999999998},"min":{"1min":0.55200000000000005,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1619999999999999,"5min":6.2939999999999996,"15min":6.2939999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.84299999999999997,"15min":0.97999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.99099999999999999,"5min":6.149,"15min":6.149},"last":0.65800000000000003}]},{"osd":1,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.84999999999999998,"15min":1.2250000000000001},"min":{"1min":0.57799999999999996,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0309999999999999,"5min":14.689,"15min":14.689},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":1.014,"15min":1.379},"min":{"1min":0.629,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.081,"5min":15.281000000000001,"15min":15.281000000000001},"last":0.89000000000000001}]},{"osd":2,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81200000000000006,"15min":0.78800000000000003},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.006,"5min":1.345,"15min":1.345},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93999999999999995,"15min":0.94699999999999995},"min":{"1min":0.66200000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.2569999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.75900000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:07:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0549999999999999,"15min":1.0840000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3069999999999999,"5min":1.538,"15min":1.538},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0089999999999999,"15min":1.087},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2110000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92800000000000005}]},{"osd":5,"last update":"Sat Sep 30 01:07:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.0960000000000001,"15min":1.163},"min":{"1min":0.89600000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.3300000000000001,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.165,"15min":1.194},"min":{"1min":0.89300000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.369,"5min":1.792,"15min":1.792},"last":1.0669999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:07:19 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1990000000000001,"15min":1.206},"min":{"1min":0.93300000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.4390000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.006},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.2130000000000001,"15min":1.2390000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3009999999999999,"5min":1.863,"15min":1.863},"last":1.149}]},{"osd":7,"last update":"Sat Sep 30 01:07:19 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.1850000000000001,"15min":1.2230000000000001},"min":{"1min":0.80000000000000004,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6830000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.242,"15min":1.246},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.8180000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.1839999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5612,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366228,"statfs":{"total":96636764160,"available":96631017472,"internally_reserved":0,"allocated":364544,"data_stored":179676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70399999999999996,"15min":0.68400000000000005},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.1220000000000001,"5min":1.409,"15min":1.409},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77400000000000002,"15min":0.80600000000000005},"min":{"1min":0.59199999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2050000000000001,"5min":1.254,"15min":1.254},"last":1.1259999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.78400000000000003,"15min":0.80500000000000005},"min":{"1min":0.57099999999999995,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0760000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82299999999999995,"15min":0.85299999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.03,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.80400000000000005}]},{"osd":3,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79800000000000004,"15min":0.80200000000000005},"min":{"1min":0.60399999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4810000000000001,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.95899999999999996,"15min":0.98999999999999999},"min":{"1min":0.58899999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2210000000000001,"5min":2.016,"15min":2.016},"last":1.0740000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.0880000000000001,"15min":1.1020000000000001},"min":{"1min":0.86799999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.506,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.03},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0880000000000001,"15min":1.129},"min":{"1min":0.78700000000000003,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.6719999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.91900000000000004}]},{"osd":5,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.125,"15min":1.1220000000000001},"min":{"1min":0.85499999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.7330000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.143,"15min":1.1850000000000001},"min":{"1min":0.753,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.675,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.175}]},{"osd":6,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1559999999999999,"15min":1.208},"min":{"1min":0.80600000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.5509999999999999,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.105},{"interface":"front","average":{"1min":1.224,"5min":1.254,"15min":1.2869999999999999},"min":{"1min":0.94199999999999995,"5min":0.748,"15min":0.748},"max":{"1min":1.6299999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.2030000000000001}]},{"osd":7,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.274,"15min":1.294},"min":{"1min":0.86399999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.736,"5min":2.137,"15min":2.137},"last":1.153},{"interface":"front","average":{"1min":1.264,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.97399999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":1.2290000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5684,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94366156,"statfs":{"total":96636764160,"available":96630943744,"internally_reserved":0,"allocated":372736,"data_stored":180576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67300000000000004,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97599999999999998,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.78000000000000003,"15min":0.81999999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1659999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.63100000000000001}]},{"osd":2,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96799999999999997,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.87,"15min":0.90300000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2929999999999999,"5min":1.726,"15min":1.726},"last":0.82399999999999995}]},{"osd":3,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.89800000000000002,"15min":0.89600000000000002},"min":{"1min":0.63400000000000001,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.071,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.878,"15min":0.90200000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.095,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.94499999999999995}]},{"osd":4,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.089,"15min":1.143},"min":{"1min":0.80600000000000005,"5min":0.747,"15min":0.747},"max":{"1min":1.5429999999999999,"5min":1.893,"15min":1.893},"last":1.03},{"interface":"front","average":{"1min":1.01,"5min":1.1020000000000001,"15min":1.125},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.335,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.056}]},{"osd":5,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0980000000000001,"15min":1.119},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.595,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.1040000000000001,"15min":1.1279999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3819999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.98399999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.131,"15min":1.163},"min":{"1min":0.80700000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.498,"5min":1.508,"15min":1.508},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.2150000000000001,"15min":1.246},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.7430000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.0800000000000001}]},{"osd":7,"last update":"Sat Sep 30 01:07:15 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.248,"15min":1.3},"min":{"1min":0.90000000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.444,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2270000000000001,"15min":1.2849999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.504,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.1060000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6184,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94365656,"statfs":{"total":96636764160,"available":96630431744,"internally_reserved":0,"allocated":425984,"data_stored":709504,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 01:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66400000000000003,"15min":0.70199999999999996},"min":{"1min":0.52000000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1140000000000001,"5min":1.232,"15min":1.232},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81299999999999994,"15min":0.81999999999999995},"min":{"1min":0.64200000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.165,"5min":1.377,"15min":1.377},"last":0.73299999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.84299999999999997,"15min":0.873},"min":{"1min":0.58699999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3069999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.745,"15min":0.745},"min":{"1min":0.55700000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.032,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.82399999999999995}]},{"osd":3,"last update":"Sat Sep 30 01:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82599999999999996,"15min":0.83199999999999996},"min":{"1min":0.59999999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96699999999999997,"15min":0.98599999999999999},"min":{"1min":0.80300000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.335,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0720000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.0069999999999999,"15min":1.0449999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.4319999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.0900000000000001,"15min":1.1319999999999999},"min":{"1min":0.82699999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.46,"5min":1.655,"15min":1.655},"last":1.1919999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.073,"15min":1.103},"min":{"1min":0.95199999999999996,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3640000000000001,"5min":1.482,"15min":1.482},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.1739999999999999,"15min":1.204},"min":{"1min":0.79600000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.5700000000000001,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.109}]},{"osd":6,"last update":"Sat Sep 30 01:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1180000000000001,"15min":1.1339999999999999},"min":{"1min":0.82399999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.383,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.212,"15min":1.256},"min":{"1min":0.86599999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.014}]},{"osd":7,"last update":"Sat Sep 30 01:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.244,"15min":1.3},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.6479999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.173,"5min":1.206,"15min":1.244},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.482,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.2490000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5931,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:07:39.811 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-09-30T01:07:40.130 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1895, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1472, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph_manager.py", line 3189, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-09-30T01:07:40.131 INFO:teuthology.misc:Shutting down mds daemons... 2023-09-30T01:07:40.131 INFO:teuthology.misc:Shutting down osd daemons... 2023-09-30T01:07:40.132 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-30T01:07:40.132 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.199 INFO:tasks.ceph.osd.0:Stopped 2023-09-30T01:07:40.200 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-30T01:07:40.200 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.281 INFO:tasks.ceph.osd.1:Stopped 2023-09-30T01:07:40.281 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-30T01:07:40.282 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.346 INFO:tasks.ceph.osd.2:Stopped 2023-09-30T01:07:40.346 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-30T01:07:40.346 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.435 INFO:tasks.ceph.osd.3:Stopped 2023-09-30T01:07:40.435 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-09-30T01:07:40.435 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.515 INFO:tasks.ceph.osd.4:Stopped 2023-09-30T01:07:40.515 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-09-30T01:07:40.515 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.593 INFO:tasks.ceph.osd.5:Stopped 2023-09-30T01:07:40.593 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-09-30T01:07:40.593 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.678 INFO:tasks.ceph.osd.6:Stopped 2023-09-30T01:07:40.678 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-09-30T01:07:40.679 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.777 INFO:tasks.ceph.osd.7:Stopped 2023-09-30T01:07:40.777 INFO:teuthology.misc:Shutting down mgr daemons... 2023-09-30T01:07:40.777 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-09-30T01:07:40.777 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.807 INFO:tasks.ceph.mgr.y:Stopped 2023-09-30T01:07:40.885 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-30T01:07:40.885 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.916 INFO:tasks.ceph.mgr.x:Stopped 2023-09-30T01:07:40.917 INFO:teuthology.misc:Shutting down mon daemons... 2023-09-30T01:07:40.917 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-30T01:07:40.917 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.938 INFO:tasks.ceph.mon.a:Stopped 2023-09-30T01:07:40.938 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-30T01:07:40.938 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.955 INFO:tasks.ceph.mon.c:Stopped 2023-09-30T01:07:40.955 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-30T01:07:40.955 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:07:40.974 INFO:tasks.ceph.mon.b:Stopped 2023-09-30T01:07:40.975 INFO:tasks.ceph:Checking cluster log for badness... 2023-09-30T01:07:40.975 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(REQUEST_STUCK\)' | egrep -v 'failed to encode map' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'must scrub before tier agent can activate' | head -n 1 2023-09-30T01:07:40.995 INFO:teuthology.orchestra.run.smithi037.stdout:1696035719.6175494 mgr.x (mgr.4107) 1 : cluster [ERR] Unhandled exception from module 'pg_autoscaler' while running on mgr.x: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-30T01:07:40.996 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-09-30T01:07:40.996 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(REQUEST_STUCK\)' | egrep -v 'failed to encode map' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'must scrub before tier agent can activate' | head -n 1 2023-09-30T01:07:41.060 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(REQUEST_STUCK\)' | egrep -v 'failed to encode map' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'must scrub before tier agent can activate' | head -n 1 2023-09-30T01:07:41.073 INFO:teuthology.orchestra.run.smithi037.stdout:1696035719.6175494 mgr.x (mgr.4107) 1 : cluster [ERR] Unhandled exception from module 'pg_autoscaler' while running on mgr.x: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-30T01:07:41.074 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi037.front.sepia.ceph.com 2023-09-30T01:07:41.075 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-09-30T01:07:41.299 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi037.front.sepia.ceph.com 2023-09-30T01:07:41.299 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-09-30T01:07:41.553 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi037.front.sepia.ceph.com 2023-09-30T01:07:41.554 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-09-30T01:07:41.718 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi037.front.sepia.ceph.com 2023-09-30T01:07:41.718 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-09-30T01:07:41.854 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi050.front.sepia.ceph.com 2023-09-30T01:07:41.854 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-09-30T01:07:42.102 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi050.front.sepia.ceph.com 2023-09-30T01:07:42.103 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-09-30T01:07:42.278 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi050.front.sepia.ceph.com 2023-09-30T01:07:42.278 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-09-30T01:07:42.454 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi050.front.sepia.ceph.com 2023-09-30T01:07:42.454 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-09-30T01:07:42.779 INFO:tasks.ceph:Archiving mon data... 2023-09-30T01:07:42.780 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821/data/mon.a.tgz 2023-09-30T01:07:42.780 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2023-09-30T01:07:42.787 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.16GKmxStK0 2023-09-30T01:07:42.787 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.16GKmxStK0 2023-09-30T01:07:42.864 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.16GKmxStK0 2023-09-30T01:07:42.937 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.16GKmxStK0 is 294KB 2023-09-30T01:07:42.958 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.16GKmxStK0 2023-09-30T01:07:42.964 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821/data/mon.c.tgz 2023-09-30T01:07:42.964 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2023-09-30T01:07:43.014 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.ki8eTbCiM7 2023-09-30T01:07:43.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ki8eTbCiM7 2023-09-30T01:07:43.093 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.ki8eTbCiM7 2023-09-30T01:07:43.113 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.ki8eTbCiM7 is 294KB 2023-09-30T01:07:43.131 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.ki8eTbCiM7 2023-09-30T01:07:43.137 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821/data/mon.b.tgz 2023-09-30T01:07:43.137 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2023-09-30T01:07:43.144 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.77sfdKbUaX 2023-09-30T01:07:43.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.77sfdKbUaX 2023-09-30T01:07:43.263 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.77sfdKbUaX 2023-09-30T01:07:43.288 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.77sfdKbUaX is 294KB 2023-09-30T01:07:43.349 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.77sfdKbUaX 2023-09-30T01:07:43.356 INFO:tasks.ceph:Cleaning ceph cluster... 2023-09-30T01:07:43.356 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-30T01:07:43.358 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-30T01:07:43.373 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-09-30T01:07:43.374 DEBUG:teuthology.orchestra.run.smithi037:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-30T01:07:43.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-30T01:07:43.403 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-30T01:07:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-30T01:07:43.421 INFO:tasks.ceph:Archiving crash dumps... 2023-09-30T01:07:43.423 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821/remote/smithi037/crash 2023-09-30T01:07:43.424 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-30T01:07:43.481 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821/remote/smithi050/crash 2023-09-30T01:07:43.482 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-30T01:07:43.502 INFO:tasks.ceph:Compressing logs... 2023-09-30T01:07:43.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T01:07:43.505 DEBUG:teuthology.orchestra.run.smithi050:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T01:07:44.615 INFO:tasks.ceph:Archiving logs... 2023-09-30T01:07:44.615 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821/remote/smithi037/log 2023-09-30T01:07:44.616 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T01:07:44.919 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821/remote/smithi050/log 2023-09-30T01:07:44.920 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T01:07:45.054 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1895, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1472, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph_manager.py", line 3189, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-09-30T01:07:45.131 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c667e809543543ad9f349fbaa72fd2d7 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1895, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1472, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph_manager.py", line 3189, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-09-30T01:07:45.133 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-09-30T01:07:45.149 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-30T01:07:45.157 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-30T01:07:45.157 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T01:07:45.171 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T01:07:45.238 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-30T01:07:45.248 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-30T01:07:45.248 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T01:07:45.250 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T01:07:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: remote refid st t when poll reach delay offset jitter 2023-09-30T01:07:45.271 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================== 2023-09-30T01:07:45.271 INFO:teuthology.orchestra.run.smithi037.stdout:*hv01.front.sepi 44.4.53.2 2 u 54 64 377 0.078 1.146 0.813 2023-09-30T01:07:45.271 INFO:teuthology.orchestra.run.smithi037.stdout:+hv02.front.sepi 63.231.80.2 3 u 47 64 377 0.065 2.158 1.182 2023-09-30T01:07:45.271 INFO:teuthology.orchestra.run.smithi037.stdout:+hv03.front.sepi 74.6.168.72 3 u 46 64 377 0.082 1.215 1.164 2023-09-30T01:07:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T01:07:45.273 INFO:teuthology.orchestra.run.smithi050.stdout: remote refid st t when poll reach delay offset jitter 2023-09-30T01:07:45.273 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================== 2023-09-30T01:07:45.273 INFO:teuthology.orchestra.run.smithi050.stdout: hv01.front.sepi 44.4.53.2 2 u 46 64 377 0.095 -0.024 0.660 2023-09-30T01:07:45.273 INFO:teuthology.orchestra.run.smithi050.stdout:*hv02.front.sepi 63.231.80.2 3 u 50 64 377 0.085 0.040 0.650 2023-09-30T01:07:45.274 INFO:teuthology.orchestra.run.smithi050.stdout: hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.084 -1.449 0.282 2023-09-30T01:07:45.274 INFO:teuthology.orchestra.run.smithi050.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T01:07:45.275 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-30T01:07:45.283 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-30T01:07:45.284 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-30T01:07:45.292 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-30T01:07:45.300 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-30T01:07:45.309 INFO:teuthology.task.internal:Duration was 868.245787 seconds 2023-09-30T01:07:45.310 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-30T01:07:45.318 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-30T01:07:45.319 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T01:07:45.322 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T01:07:45.374 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-30T01:07:45.375 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2023-09-30T01:07:45.375 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T01:07:45.388 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2023-09-30T01:07:45.389 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T01:07:45.401 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-30T01:07:45.401 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T01:07:45.403 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T01:07:45.434 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-30T01:07:45.434 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T01:07:45.474 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T01:07:45.540 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-30T01:07:45.552 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-30T01:07:45.553 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T01:07:45.568 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T01:07:45.584 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-30T01:07:45.594 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T01:07:45.614 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T01:07:45.630 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2023-09-30T01:07:45.644 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2023-09-30T01:07:45.667 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T01:07:45.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:07:45.673 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T01:07:45.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:07:45.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-30T01:07:45.691 INFO:teuthology.task.internal:Transferring archived files... 2023-09-30T01:07:45.691 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821/remote/smithi037 2023-09-30T01:07:45.692 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-30T01:07:45.715 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821/remote/smithi050 2023-09-30T01:07:45.715 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-30T01:07:45.747 INFO:teuthology.task.internal:Removing archive directory... 2023-09-30T01:07:45.747 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-30T01:07:45.754 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-30T01:07:45.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-30T01:07:45.772 INFO:teuthology.task.internal:Not uploading archives. 2023-09-30T01:07:45.772 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-30T01:07:45.782 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-30T01:07:45.782 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T01:07:45.785 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T01:07:45.790 INFO:teuthology.orchestra.run.smithi037.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 01:07 /home/ubuntu/cephtest 2023-09-30T01:07:45.810 INFO:teuthology.orchestra.run.smithi050.stdout: 924113 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 01:07 /home/ubuntu/cephtest 2023-09-30T01:07:45.812 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-30T01:07:45.822 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-30T01:07:45.844 INFO:teuthology.nuke:Checking targets against current locks 2023-09-30T01:07:45.871 DEBUG:teuthology.nuke:shortname: smithi037 2023-09-30T01:07:45.872 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T01:07:45.898 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 00:44:10.785945', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-09-30T01:07:45.902 DEBUG:teuthology.nuke:shortname: smithi050 2023-09-30T01:07:45.902 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T01:07:45.927 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 00:44:10.783937', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-09-30T01:07:45.947 INFO:teuthology.orchestra.console.smithi037:Power off 2023-09-30T01:07:45.947 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T01:07:45.974 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2023-09-30T01:07:45.974 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:07:45.991 INFO:teuthology.orchestra.console.smithi050:Power off 2023-09-30T01:07:45.992 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T01:07:46.002 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2023-09-30T01:07:46.015 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off 2023-09-30T01:07:46.016 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:07:46.048 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2023-09-30T01:07:50.003 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:07:50.048 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:07:50.113 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2023-09-30T01:07:50.159 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2023-09-30T01:07:54.116 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:07:54.160 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:07:54.228 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2023-09-30T01:07:54.271 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2023-09-30T01:07:54.329 INFO:teuthology.orchestra.console.smithi037:Power off completed 2023-09-30T01:07:58.274 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:07:58.384 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2023-09-30T01:07:58.484 INFO:teuthology.orchestra.console.smithi050:Power off completed 2023-09-30T01:07:58.590 INFO:teuthology.run:Summary data: description: rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache} duration: 868.2457873821259 failure_reason: '"1696035719.6175494 mgr.x (mgr.4107) 1 : cluster [ERR] Unhandled exception from module ''pg_autoscaler'' while running on mgr.x: ''PgAutoscaler'' object has no attribute ''noautoscale''" in cluster log' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c667e809543543ad9f349fbaa72fd2d7 status: fail success: false 2023-09-30T01:07:58.590 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T01:07:58.665 INFO:teuthology.run:FAIL