2023-09-30T00:44:18.231 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-30T00:44:18.231 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794 --owner scheduled_yuriw@teuthology --name yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi --verbose --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-snaps} -- /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794/orig.config.yaml 2023-09-30T00:44:18.276 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T00:44:18.369 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794 branch: wip-yuri4-testing-2023-09-29-0928-quincy description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-snaps} email: yweinste@redhat.com first_in_suite: false job_id: '7406794' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri4-testing-2023-09-29-0928-quincy ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 ms type: async osd_max_pg_log_entries: 2 osd_min_pg_log_entries: 1 osd_pg_log_trim_min: 0 osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore odsync write: true mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd backoff on degraded: true osd backoff on peering: true osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 2 osd max markdown count: 1000 osd max scrubs: 3 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 0.5 osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - must scrub before tier agent can activate sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 min_in: 4 workunit: branch: wip-yuri4-testing-2023-09-29-0928-quincy sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 9810 sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri4-testing-2023-09-29-0928-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 targets: smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 2 chance_pgpnum_fix: 1 timeout: 1200 - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache target_max_objects 250 - sudo ceph osd pool set cache min_read_recency_for_promote 2 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-29_19:38:56 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-30T00:44:18.369 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa; will attempt to use it 2023-09-30T00:44:18.370 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks 2023-09-30T00:44:18.370 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-30T00:44:18.372 INFO:teuthology.task.internal:Checking packages... 2023-09-30T00:44:18.395 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '93dce5e17918f8aca65993a7a2dc40233a47b481' 2023-09-30T00:44:18.396 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T00:44:18.396 INFO:teuthology.packaging:ref: None 2023-09-30T00:44:18.396 INFO:teuthology.packaging:tag: None 2023-09-30T00:44:18.396 INFO:teuthology.packaging:branch: wip-yuri4-testing-2023-09-29-0928-quincy 2023-09-30T00:44:18.396 INFO:teuthology.packaging:sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-30T00:44:18.396 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri4-testing-2023-09-29-0928-quincy 2023-09-30T00:44:18.622 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:44:18.623 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-30T00:44:18.629 INFO:teuthology.task.internal:no buildpackages task found 2023-09-30T00:44:18.630 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-30T00:44:18.661 INFO:teuthology.task.internal:Saving configuration 2023-09-30T00:44:18.678 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-30T00:44:18.685 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T00:44:18.711 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 00:38:16.726695', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-09-30T00:44:18.734 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 00:38:16.728600', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-09-30T00:44:18.735 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-30T00:44:18.741 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-09-30T00:44:18.741 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-09-30T00:44:18.741 INFO:teuthology.run_tasks:Running task console_log... 2023-09-30T00:44:18.834 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f20058330d0>, signals=[15]) 2023-09-30T00:44:18.835 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-30T00:44:18.841 INFO:teuthology.task.internal:Opening connections... 2023-09-30T00:44:18.841 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2023-09-30T00:44:18.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:44:18.893 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2023-09-30T00:44:18.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:44:18.943 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-30T00:44:18.950 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2023-09-30T00:44:18.974 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2023-09-30T00:44:18.974 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2023-09-30T00:44:19.020 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="Ubuntu" 2023-09-30T00:44:19.021 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-09-30T00:44:19.021 INFO:teuthology.orchestra.run.smithi069.stdout:ID=ubuntu 2023-09-30T00:44:19.021 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE=debian 2023-09-30T00:44:19.021 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-09-30T00:44:19.021 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="20.04" 2023-09-30T00:44:19.021 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-30T00:44:19.022 INFO:teuthology.orchestra.run.smithi069.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-30T00:44:19.022 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-30T00:44:19.022 INFO:teuthology.orchestra.run.smithi069.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-30T00:44:19.022 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_CODENAME=focal 2023-09-30T00:44:19.022 INFO:teuthology.orchestra.run.smithi069.stdout:UBUNTU_CODENAME=focal 2023-09-30T00:44:19.025 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2023-09-30T00:44:19.051 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2023-09-30T00:44:19.057 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2023-09-30T00:44:19.058 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2023-09-30T00:44:19.108 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="Ubuntu" 2023-09-30T00:44:19.108 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-09-30T00:44:19.108 INFO:teuthology.orchestra.run.smithi150.stdout:ID=ubuntu 2023-09-30T00:44:19.108 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE=debian 2023-09-30T00:44:19.108 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-09-30T00:44:19.108 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="20.04" 2023-09-30T00:44:19.110 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-30T00:44:19.110 INFO:teuthology.orchestra.run.smithi150.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-30T00:44:19.110 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-30T00:44:19.110 INFO:teuthology.orchestra.run.smithi150.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-30T00:44:19.111 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_CODENAME=focal 2023-09-30T00:44:19.111 INFO:teuthology.orchestra.run.smithi150.stdout:UBUNTU_CODENAME=focal 2023-09-30T00:44:19.111 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2023-09-30T00:44:19.134 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-30T00:44:19.143 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-30T00:44:19.161 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-30T00:44:19.162 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2023-09-30T00:44:19.165 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2023-09-30T00:44:19.170 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-30T00:44:19.186 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-30T00:44:19.187 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2023-09-30T00:44:19.189 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2023-09-30T00:44:19.235 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-30T00:44:19.293 INFO:teuthology.run_tasks:Running task kernel... 2023-09-30T00:44:19.308 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-09-30T00:44:19.308 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-09-30T00:44:19.308 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T00:44:19.309 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T00:44:19.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:44:19.314 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2023-09-30T00:44:19.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:44:19.315 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2023-09-30T00:44:19.321 INFO:teuthology.orchestra.run.smithi069.stdout:5.4.0-124-generic 2023-09-30T00:44:19.322 INFO:teuthology.task.kernel:Running kernel on smithi069: 5.4.0-124-generic 2023-09-30T00:44:19.322 DEBUG:teuthology.orchestra.run.smithi069:> sudo apt-get clean 2023-09-30T00:44:19.323 INFO:teuthology.orchestra.run.smithi150.stdout:5.4.0-124-generic 2023-09-30T00:44:19.323 INFO:teuthology.task.kernel:Running kernel on smithi150: 5.4.0-124-generic 2023-09-30T00:44:19.323 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get clean 2023-09-30T00:44:19.451 DEBUG:teuthology.orchestra.run.smithi069:> sudo apt-get update 2023-09-30T00:44:19.471 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2023-09-30T00:44:19.616 INFO:teuthology.orchestra.run.smithi069.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:44:19.626 INFO:teuthology.orchestra.run.smithi069.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-09-30T00:44:19.626 INFO:teuthology.orchestra.run.smithi069.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-09-30T00:44:19.632 INFO:teuthology.orchestra.run.smithi150.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-09-30T00:44:19.688 INFO:teuthology.orchestra.run.smithi069.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-09-30T00:44:19.726 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:44:19.808 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-09-30T00:44:19.932 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [652 kB] 2023-09-30T00:44:19.995 INFO:teuthology.orchestra.run.smithi069.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,852 kB] 2023-09-30T00:44:20.014 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,464 kB] 2023-09-30T00:44:20.060 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [385 kB] 2023-09-30T00:44:20.061 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-09-30T00:44:20.061 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,218 kB] 2023-09-30T00:44:20.072 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-09-30T00:44:20.072 INFO:teuthology.orchestra.run.smithi150.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [309 kB] 2023-09-30T00:44:20.073 INFO:teuthology.orchestra.run.smithi150.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-09-30T00:44:20.073 INFO:teuthology.orchestra.run.smithi150.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [886 kB] 2023-09-30T00:44:20.079 INFO:teuthology.orchestra.run.smithi150.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [615 kB] 2023-09-30T00:44:20.083 INFO:teuthology.orchestra.run.smithi150.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [185 kB] 2023-09-30T00:44:20.084 INFO:teuthology.orchestra.run.smithi150.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-09-30T00:44:20.085 INFO:teuthology.orchestra.run.smithi150.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-09-30T00:44:20.085 INFO:teuthology.orchestra.run.smithi150.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-09-30T00:44:20.096 INFO:teuthology.orchestra.run.smithi150.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-09-30T00:44:20.097 INFO:teuthology.orchestra.run.smithi150.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-09-30T00:44:20.125 INFO:teuthology.orchestra.run.smithi069.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [652 kB] 2023-09-30T00:44:20.126 INFO:teuthology.orchestra.run.smithi069.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [885 kB] 2023-09-30T00:44:20.140 INFO:teuthology.orchestra.run.smithi069.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [469 kB] 2023-09-30T00:44:20.144 INFO:teuthology.orchestra.run.smithi069.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-09-30T00:44:20.144 INFO:teuthology.orchestra.run.smithi069.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.7 kB] 2023-09-30T00:44:20.145 INFO:teuthology.orchestra.run.smithi069.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,330 kB] 2023-09-30T00:44:20.152 INFO:teuthology.orchestra.run.smithi150.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-09-30T00:44:20.160 INFO:teuthology.orchestra.run.smithi069.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [325 kB] 2023-09-30T00:44:20.162 INFO:teuthology.orchestra.run.smithi069.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-09-30T00:44:20.163 INFO:teuthology.orchestra.run.smithi069.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [748 kB] 2023-09-30T00:44:20.169 INFO:teuthology.orchestra.run.smithi069.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,117 kB] 2023-09-30T00:44:20.177 INFO:teuthology.orchestra.run.smithi069.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [267 kB] 2023-09-30T00:44:20.179 INFO:teuthology.orchestra.run.smithi069.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-09-30T00:44:20.179 INFO:teuthology.orchestra.run.smithi069.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-09-30T00:44:20.179 INFO:teuthology.orchestra.run.smithi069.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.9 kB] 2023-09-30T00:44:20.180 INFO:teuthology.orchestra.run.smithi069.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-09-30T00:44:20.190 INFO:teuthology.orchestra.run.smithi069.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-09-30T00:44:20.195 INFO:teuthology.orchestra.run.smithi069.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,464 kB] 2023-09-30T00:44:20.235 INFO:teuthology.orchestra.run.smithi069.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [385 kB] 2023-09-30T00:44:20.240 INFO:teuthology.orchestra.run.smithi069.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-09-30T00:44:20.240 INFO:teuthology.orchestra.run.smithi069.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-09-30T00:44:20.241 INFO:teuthology.orchestra.run.smithi069.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,218 kB] 2023-09-30T00:44:20.249 INFO:teuthology.orchestra.run.smithi069.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-09-30T00:44:20.265 INFO:teuthology.orchestra.run.smithi069.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [309 kB] 2023-09-30T00:44:20.267 INFO:teuthology.orchestra.run.smithi069.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-09-30T00:44:20.268 INFO:teuthology.orchestra.run.smithi069.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-09-30T00:44:20.269 INFO:teuthology.orchestra.run.smithi069.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-09-30T00:44:20.269 INFO:teuthology.orchestra.run.smithi069.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-09-30T00:44:20.270 INFO:teuthology.orchestra.run.smithi069.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-09-30T00:44:20.270 INFO:teuthology.orchestra.run.smithi069.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-09-30T00:44:20.270 INFO:teuthology.orchestra.run.smithi069.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [615 kB] 2023-09-30T00:44:20.277 INFO:teuthology.orchestra.run.smithi069.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [886 kB] 2023-09-30T00:44:20.282 INFO:teuthology.orchestra.run.smithi069.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [185 kB] 2023-09-30T00:44:20.284 INFO:teuthology.orchestra.run.smithi069.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-09-30T00:44:20.284 INFO:teuthology.orchestra.run.smithi069.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-09-30T00:44:20.284 INFO:teuthology.orchestra.run.smithi069.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-09-30T00:44:20.285 INFO:teuthology.orchestra.run.smithi069.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-09-30T00:44:20.287 INFO:teuthology.orchestra.run.smithi069.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-09-30T00:44:20.294 INFO:teuthology.orchestra.run.smithi150.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,852 kB] 2023-09-30T00:44:20.673 INFO:teuthology.orchestra.run.smithi150.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [885 kB] 2023-09-30T00:44:20.702 INFO:teuthology.orchestra.run.smithi150.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [469 kB] 2023-09-30T00:44:20.716 INFO:teuthology.orchestra.run.smithi150.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-09-30T00:44:20.716 INFO:teuthology.orchestra.run.smithi150.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.7 kB] 2023-09-30T00:44:20.717 INFO:teuthology.orchestra.run.smithi150.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,330 kB] 2023-09-30T00:44:20.776 INFO:teuthology.orchestra.run.smithi150.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [325 kB] 2023-09-30T00:44:20.786 INFO:teuthology.orchestra.run.smithi150.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-09-30T00:44:20.961 INFO:teuthology.orchestra.run.smithi150.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,117 kB] 2023-09-30T00:44:21.590 INFO:teuthology.orchestra.run.smithi150.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [748 kB] 2023-09-30T00:44:21.628 INFO:teuthology.orchestra.run.smithi150.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [267 kB] 2023-09-30T00:44:21.642 INFO:teuthology.orchestra.run.smithi150.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-09-30T00:44:21.643 INFO:teuthology.orchestra.run.smithi150.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-09-30T00:44:21.643 INFO:teuthology.orchestra.run.smithi150.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.9 kB] 2023-09-30T00:44:21.645 INFO:teuthology.orchestra.run.smithi150.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-09-30T00:44:21.645 INFO:teuthology.orchestra.run.smithi150.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-09-30T00:44:21.645 INFO:teuthology.orchestra.run.smithi150.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-09-30T00:44:21.671 INFO:teuthology.orchestra.run.smithi150.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-09-30T00:44:21.673 INFO:teuthology.orchestra.run.smithi150.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-09-30T00:44:21.837 INFO:teuthology.orchestra.run.smithi150.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-09-30T00:44:22.021 INFO:teuthology.orchestra.run.smithi150.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-09-30T00:44:22.043 INFO:teuthology.orchestra.run.smithi150.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-09-30T00:44:23.851 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 17.4 MB in 3s (6,948 kB/s) 2023-09-30T00:44:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:Fetched 17.4 MB in 3s (6,893 kB/s) 2023-09-30T00:44:25.207 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-09-30T00:44:25.232 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-30T00:44:25.312 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-09-30T00:44:25.407 INFO:teuthology.orchestra.run.smithi069.stdout:Reading package lists... 2023-09-30T00:44:25.433 DEBUG:teuthology.orchestra.run.smithi069:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-30T00:44:25.508 INFO:teuthology.orchestra.run.smithi069.stdout:Reading package lists... 2023-09-30T00:44:25.512 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2023-09-30T00:44:25.512 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2023-09-30T00:44:25.698 INFO:teuthology.orchestra.run.smithi150.stdout:The following additional packages will be installed: 2023-09-30T00:44:25.698 INFO:teuthology.orchestra.run.smithi150.stdout: linux-generic linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-30T00:44:25.698 INFO:teuthology.orchestra.run.smithi150.stdout: linux-headers-generic linux-image-5.4.0-163-generic 2023-09-30T00:44:25.698 INFO:teuthology.orchestra.run.smithi150.stdout: linux-modules-5.4.0-163-generic linux-modules-extra-5.4.0-163-generic 2023-09-30T00:44:25.699 INFO:teuthology.orchestra.run.smithi150.stdout:Suggested packages: 2023-09-30T00:44:25.699 INFO:teuthology.orchestra.run.smithi150.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-09-30T00:44:25.699 INFO:teuthology.orchestra.run.smithi150.stdout:Recommended packages: 2023-09-30T00:44:25.699 INFO:teuthology.orchestra.run.smithi150.stdout: thermald 2023-09-30T00:44:25.705 INFO:teuthology.orchestra.run.smithi069.stdout:Building dependency tree... 2023-09-30T00:44:25.705 INFO:teuthology.orchestra.run.smithi069.stdout:Reading state information... 2023-09-30T00:44:25.754 INFO:teuthology.orchestra.run.smithi150.stdout:The following NEW packages will be installed: 2023-09-30T00:44:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-30T00:44:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: linux-image-5.4.0-163-generic linux-modules-5.4.0-163-generic 2023-09-30T00:44:25.754 INFO:teuthology.orchestra.run.smithi150.stdout: linux-modules-extra-5.4.0-163-generic 2023-09-30T00:44:25.755 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages will be upgraded: 2023-09-30T00:44:25.755 INFO:teuthology.orchestra.run.smithi150.stdout: linux-generic linux-headers-generic linux-image-generic 2023-09-30T00:44:25.814 INFO:teuthology.orchestra.run.smithi150.stdout:3 upgraded, 5 newly installed, 0 to remove and 263 not upgraded. 2023-09-30T00:44:25.814 INFO:teuthology.orchestra.run.smithi150.stdout:Need to get 77.1 MB of archives. 2023-09-30T00:44:25.815 INFO:teuthology.orchestra.run.smithi150.stdout:After this operation, 380 MB of additional disk space will be used. 2023-09-30T00:44:25.815 INFO:teuthology.orchestra.run.smithi150.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-163-generic amd64 5.4.0-163.180 [15.0 MB] 2023-09-30T00:44:25.879 INFO:teuthology.orchestra.run.smithi069.stdout:The following additional packages will be installed: 2023-09-30T00:44:25.879 INFO:teuthology.orchestra.run.smithi069.stdout: linux-generic linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-30T00:44:25.879 INFO:teuthology.orchestra.run.smithi069.stdout: linux-headers-generic linux-image-5.4.0-163-generic 2023-09-30T00:44:25.880 INFO:teuthology.orchestra.run.smithi069.stdout: linux-modules-5.4.0-163-generic linux-modules-extra-5.4.0-163-generic 2023-09-30T00:44:25.880 INFO:teuthology.orchestra.run.smithi069.stdout:Suggested packages: 2023-09-30T00:44:25.880 INFO:teuthology.orchestra.run.smithi069.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-09-30T00:44:25.880 INFO:teuthology.orchestra.run.smithi069.stdout:Recommended packages: 2023-09-30T00:44:25.880 INFO:teuthology.orchestra.run.smithi069.stdout: thermald 2023-09-30T00:44:25.932 INFO:teuthology.orchestra.run.smithi069.stdout:The following NEW packages will be installed: 2023-09-30T00:44:25.932 INFO:teuthology.orchestra.run.smithi069.stdout: linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-30T00:44:25.932 INFO:teuthology.orchestra.run.smithi069.stdout: linux-image-5.4.0-163-generic linux-modules-5.4.0-163-generic 2023-09-30T00:44:25.933 INFO:teuthology.orchestra.run.smithi069.stdout: linux-modules-extra-5.4.0-163-generic 2023-09-30T00:44:25.933 INFO:teuthology.orchestra.run.smithi069.stdout:The following packages will be upgraded: 2023-09-30T00:44:25.934 INFO:teuthology.orchestra.run.smithi069.stdout: linux-generic linux-headers-generic linux-image-generic 2023-09-30T00:44:25.997 INFO:teuthology.orchestra.run.smithi069.stdout:3 upgraded, 5 newly installed, 0 to remove and 263 not upgraded. 2023-09-30T00:44:25.998 INFO:teuthology.orchestra.run.smithi069.stdout:Need to get 77.1 MB of archives. 2023-09-30T00:44:25.998 INFO:teuthology.orchestra.run.smithi069.stdout:After this operation, 380 MB of additional disk space will be used. 2023-09-30T00:44:25.998 INFO:teuthology.orchestra.run.smithi069.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-163-generic amd64 5.4.0-163.180 [15.0 MB] 2023-09-30T00:44:26.157 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-163-generic amd64 5.4.0-163.180 [10.5 MB] 2023-09-30T00:44:26.306 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-163-generic amd64 5.4.0-163.180 [39.2 MB] 2023-09-30T00:44:26.559 INFO:teuthology.orchestra.run.smithi069.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-163-generic amd64 5.4.0-163.180 [10.5 MB] 2023-09-30T00:44:26.737 INFO:teuthology.orchestra.run.smithi069.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-163-generic amd64 5.4.0-163.180 [39.2 MB] 2023-09-30T00:44:27.042 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.163.160 [1,904 B] 2023-09-30T00:44:27.043 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.163.160 [2,604 B] 2023-09-30T00:44:27.043 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163 all 5.4.0-163.180 [11.0 MB] 2023-09-30T00:44:27.380 INFO:teuthology.orchestra.run.smithi069.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.163.160 [1,904 B] 2023-09-30T00:44:27.380 INFO:teuthology.orchestra.run.smithi069.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.163.160 [2,604 B] 2023-09-30T00:44:27.381 INFO:teuthology.orchestra.run.smithi069.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163 all 5.4.0-163.180 [11.0 MB] 2023-09-30T00:44:27.487 INFO:teuthology.orchestra.run.smithi069.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163-generic amd64 5.4.0-163.180 [1,358 kB] 2023-09-30T00:44:27.501 INFO:teuthology.orchestra.run.smithi069.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.163.160 [2,468 B] 2023-09-30T00:44:27.533 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163-generic amd64 5.4.0-163.180 [1,358 kB] 2023-09-30T00:44:27.551 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.163.160 [2,468 B] 2023-09-30T00:44:28.184 INFO:teuthology.orchestra.run.smithi069.stdout:Fetched 77.1 MB in 2s (49.4 MB/s) 2023-09-30T00:44:28.231 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 77.1 MB in 2s (43.1 MB/s) 2023-09-30T00:44:28.379 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package linux-modules-5.4.0-163-generic. 2023-09-30T00:44:28.400 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-modules-5.4.0-163-generic. 2023-09-30T00:44:30.132 INFO:teuthology.orchestra.run.smithi150.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-09-30T00:44:30.137 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../0-linux-modules-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:44:30.145 INFO:teuthology.orchestra.run.smithi069.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-09-30T00:44:30.150 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../0-linux-modules-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:44:30.214 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:30.234 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:32.086 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package linux-image-5.4.0-163-generic. 2023-09-30T00:44:32.100 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../1-linux-image-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:44:32.139 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-image-5.4.0-163-generic. 2023-09-30T00:44:32.156 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../1-linux-image-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:44:32.208 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:32.258 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:33.314 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-163-generic. 2023-09-30T00:44:33.327 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:44:33.342 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-163-generic. 2023-09-30T00:44:33.356 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:44:33.372 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:33.393 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../3-linux-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:44:38.175 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../3-linux-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:44:38.264 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:44:38.300 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking linux-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:44:38.627 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:44:38.632 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:44:38.776 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-image-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:44:38.778 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking linux-image-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:44:39.096 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-headers-5.4.0-163. 2023-09-30T00:44:39.113 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../5-linux-headers-5.4.0-163_5.4.0-163.180_all.deb ... 2023-09-30T00:44:39.115 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package linux-headers-5.4.0-163. 2023-09-30T00:44:39.131 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../5-linux-headers-5.4.0-163_5.4.0-163.180_all.deb ... 2023-09-30T00:44:39.154 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-30T00:44:39.173 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-30T00:44:44.550 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-headers-5.4.0-163-generic. 2023-09-30T00:44:44.572 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../6-linux-headers-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:44:44.617 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:44.782 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package linux-headers-5.4.0-163-generic. 2023-09-30T00:44:44.804 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../6-linux-headers-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-30T00:44:44.849 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:47.295 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:44:47.422 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:44:47.516 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.163.160_amd64.deb ... 2023-09-30T00:44:47.644 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking linux-headers-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-30T00:44:47.764 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:47.974 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:48.986 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-30T00:44:49.112 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:49.191 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-30T00:44:49.317 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:50.270 INFO:teuthology.orchestra.run.smithi150.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:44:50.270 INFO:teuthology.orchestra.run.smithi150.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-163-generic 2023-09-30T00:44:50.414 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:50.460 INFO:teuthology.orchestra.run.smithi069.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:44:50.460 INFO:teuthology.orchestra.run.smithi069.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-163-generic 2023-09-30T00:44:50.619 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:50.674 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:50.745 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:51.930 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up linux-image-generic (5.4.0.163.160) ... 2023-09-30T00:44:52.056 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up linux-headers-generic (5.4.0.163.160) ... 2023-09-30T00:44:52.207 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up linux-generic (5.4.0.163.160) ... 2023-09-30T00:44:52.334 INFO:teuthology.orchestra.run.smithi069.stdout:Processing triggers for linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:52.492 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-image-generic (5.4.0.163.160) ... 2023-09-30T00:44:52.560 INFO:teuthology.orchestra.run.smithi069.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-09-30T00:44:52.560 INFO:teuthology.orchestra.run.smithi069.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:44:52.618 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-generic (5.4.0.163.160) ... 2023-09-30T00:44:52.744 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-generic (5.4.0.163.160) ... 2023-09-30T00:44:52.871 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-30T00:44:52.941 INFO:teuthology.orchestra.run.smithi150.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-09-30T00:44:52.941 INFO:teuthology.orchestra.run.smithi150.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:45:14.774 INFO:teuthology.orchestra.run.smithi150.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-09-30T00:45:14.774 INFO:teuthology.orchestra.run.smithi150.stdout:Sourcing file `/etc/default/grub' 2023-09-30T00:45:14.789 INFO:teuthology.orchestra.run.smithi150.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-30T00:45:14.820 INFO:teuthology.orchestra.run.smithi150.stdout:Generating grub configuration file ... 2023-09-30T00:45:15.115 INFO:teuthology.orchestra.run.smithi069.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-09-30T00:45:15.115 INFO:teuthology.orchestra.run.smithi069.stdout:Sourcing file `/etc/default/grub' 2023-09-30T00:45:15.121 INFO:teuthology.orchestra.run.smithi069.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-30T00:45:15.132 INFO:teuthology.orchestra.run.smithi069.stdout:Generating grub configuration file ... 2023-09-30T00:45:15.168 INFO:teuthology.orchestra.run.smithi150.stdout:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:45:15.197 INFO:teuthology.orchestra.run.smithi150.stdout:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:45:15.487 INFO:teuthology.orchestra.run.smithi069.stdout:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:45:15.509 INFO:teuthology.orchestra.run.smithi069.stdout:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:45:15.567 INFO:teuthology.orchestra.run.smithi150.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-30T00:45:15.579 INFO:teuthology.orchestra.run.smithi150.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-30T00:45:15.891 INFO:teuthology.orchestra.run.smithi069.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-30T00:45:15.903 INFO:teuthology.orchestra.run.smithi069.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-30T00:45:18.144 INFO:teuthology.orchestra.run.smithi069.stdout:done 2023-09-30T00:45:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:done 2023-09-30T00:45:18.566 DEBUG:teuthology.orchestra.run.smithi069:> dpkg -s linux-image-generic 2023-09-30T00:45:18.617 INFO:teuthology.orchestra.run.smithi069.stdout:Package: linux-image-generic 2023-09-30T00:45:18.618 INFO:teuthology.orchestra.run.smithi069.stdout:Status: install ok installed 2023-09-30T00:45:18.618 INFO:teuthology.orchestra.run.smithi069.stdout:Priority: optional 2023-09-30T00:45:18.618 INFO:teuthology.orchestra.run.smithi069.stdout:Section: kernel 2023-09-30T00:45:18.618 INFO:teuthology.orchestra.run.smithi069.stdout:Installed-Size: 20 2023-09-30T00:45:18.618 INFO:teuthology.orchestra.run.smithi069.stdout:Maintainer: Ubuntu Kernel Team 2023-09-30T00:45:18.618 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: amd64 2023-09-30T00:45:18.618 INFO:teuthology.orchestra.run.smithi069.stdout:Source: linux-meta 2023-09-30T00:45:18.618 INFO:teuthology.orchestra.run.smithi069.stdout:Version: 5.4.0.163.160 2023-09-30T00:45:18.618 INFO:teuthology.orchestra.run.smithi069.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-30T00:45:18.618 INFO:teuthology.orchestra.run.smithi069.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-30T00:45:18.618 INFO:teuthology.orchestra.run.smithi069.stdout:Recommends: thermald 2023-09-30T00:45:18.619 INFO:teuthology.orchestra.run.smithi069.stdout:Description: Generic Linux kernel image 2023-09-30T00:45:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: This package will always depend on the latest generic kernel image 2023-09-30T00:45:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: available. 2023-09-30T00:45:18.619 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-163-generic 2023-09-30T00:45:18.619 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-30T00:45:18.619 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-30T00:45:18.619 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-09-30T00:45:18.619 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi069.front.sepia.ceph.com, path=None, version=distro) 2023-09-30T00:45:18.620 DEBUG:teuthology.orchestra.run.smithi069:> sudo apt-get clean 2023-09-30T00:45:18.701 DEBUG:teuthology.orchestra.run.smithi150:> dpkg -s linux-image-generic 2023-09-30T00:45:18.702 DEBUG:teuthology.orchestra.run.smithi069:> sudo apt-get update 2023-09-30T00:45:18.755 INFO:teuthology.orchestra.run.smithi150.stdout:Package: linux-image-generic 2023-09-30T00:45:18.755 INFO:teuthology.orchestra.run.smithi150.stdout:Status: install ok installed 2023-09-30T00:45:18.755 INFO:teuthology.orchestra.run.smithi150.stdout:Priority: optional 2023-09-30T00:45:18.755 INFO:teuthology.orchestra.run.smithi150.stdout:Section: kernel 2023-09-30T00:45:18.756 INFO:teuthology.orchestra.run.smithi150.stdout:Installed-Size: 20 2023-09-30T00:45:18.756 INFO:teuthology.orchestra.run.smithi150.stdout:Maintainer: Ubuntu Kernel Team 2023-09-30T00:45:18.756 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: amd64 2023-09-30T00:45:18.756 INFO:teuthology.orchestra.run.smithi150.stdout:Source: linux-meta 2023-09-30T00:45:18.756 INFO:teuthology.orchestra.run.smithi150.stdout:Version: 5.4.0.163.160 2023-09-30T00:45:18.756 INFO:teuthology.orchestra.run.smithi150.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-30T00:45:18.756 INFO:teuthology.orchestra.run.smithi150.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-30T00:45:18.756 INFO:teuthology.orchestra.run.smithi150.stdout:Recommends: thermald 2023-09-30T00:45:18.756 INFO:teuthology.orchestra.run.smithi150.stdout:Description: Generic Linux kernel image 2023-09-30T00:45:18.756 INFO:teuthology.orchestra.run.smithi150.stdout: This package will always depend on the latest generic kernel image 2023-09-30T00:45:18.756 INFO:teuthology.orchestra.run.smithi150.stdout: available. 2023-09-30T00:45:18.757 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-163-generic 2023-09-30T00:45:18.757 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-30T00:45:18.757 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-30T00:45:18.757 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-09-30T00:45:18.757 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=None, version=distro) 2023-09-30T00:45:18.757 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get clean 2023-09-30T00:45:18.834 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2023-09-30T00:45:18.835 INFO:teuthology.orchestra.run.smithi069.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-30T00:45:18.969 INFO:teuthology.orchestra.run.smithi069.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:45:19.050 INFO:teuthology.orchestra.run.smithi069.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-30T00:45:19.095 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:45:19.095 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-30T00:45:19.132 INFO:teuthology.orchestra.run.smithi069.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-30T00:45:19.165 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-30T00:45:19.254 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-30T00:45:20.553 INFO:teuthology.orchestra.run.smithi069.stdout:Reading package lists... 2023-09-30T00:45:20.577 DEBUG:teuthology.orchestra.run.smithi069:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-30T00:45:20.647 INFO:teuthology.orchestra.run.smithi069.stdout:Reading package lists... 2023-09-30T00:45:20.839 INFO:teuthology.orchestra.run.smithi069.stdout:Building dependency tree... 2023-09-30T00:45:20.840 INFO:teuthology.orchestra.run.smithi069.stdout:Reading state information... 2023-09-30T00:45:20.846 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-09-30T00:45:20.872 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-30T00:45:20.942 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-09-30T00:45:21.041 INFO:teuthology.orchestra.run.smithi069.stdout:linux-image-generic is already the newest version (5.4.0.163.160). 2023-09-30T00:45:21.041 INFO:teuthology.orchestra.run.smithi069.stdout:0 upgraded, 0 newly installed, 0 to remove and 263 not upgraded. 2023-09-30T00:45:21.043 DEBUG:teuthology.orchestra.run.smithi069:> dpkg -s linux-image-generic 2023-09-30T00:45:21.063 INFO:teuthology.orchestra.run.smithi069.stdout:Package: linux-image-generic 2023-09-30T00:45:21.063 INFO:teuthology.orchestra.run.smithi069.stdout:Status: install ok installed 2023-09-30T00:45:21.063 INFO:teuthology.orchestra.run.smithi069.stdout:Priority: optional 2023-09-30T00:45:21.063 INFO:teuthology.orchestra.run.smithi069.stdout:Section: kernel 2023-09-30T00:45:21.063 INFO:teuthology.orchestra.run.smithi069.stdout:Installed-Size: 20 2023-09-30T00:45:21.063 INFO:teuthology.orchestra.run.smithi069.stdout:Maintainer: Ubuntu Kernel Team 2023-09-30T00:45:21.063 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: amd64 2023-09-30T00:45:21.063 INFO:teuthology.orchestra.run.smithi069.stdout:Source: linux-meta 2023-09-30T00:45:21.064 INFO:teuthology.orchestra.run.smithi069.stdout:Version: 5.4.0.163.160 2023-09-30T00:45:21.064 INFO:teuthology.orchestra.run.smithi069.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-30T00:45:21.064 INFO:teuthology.orchestra.run.smithi069.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-30T00:45:21.064 INFO:teuthology.orchestra.run.smithi069.stdout:Recommends: thermald 2023-09-30T00:45:21.064 INFO:teuthology.orchestra.run.smithi069.stdout:Description: Generic Linux kernel image 2023-09-30T00:45:21.064 INFO:teuthology.orchestra.run.smithi069.stdout: This package will always depend on the latest generic kernel image 2023-09-30T00:45:21.064 INFO:teuthology.orchestra.run.smithi069.stdout: available. 2023-09-30T00:45:21.064 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2023-09-30T00:45:21.068 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.nK2jDAtgu7 2023-09-30T00:45:21.069 DEBUG:teuthology.orchestra.run.smithi069:> sudo cp /boot/grub/grub.cfg /tmp/tmp.nK2jDAtgu7 2023-09-30T00:45:21.125 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.nK2jDAtgu7 2023-09-30T00:45:21.145 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2023-09-30T00:45:21.146 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2023-09-30T00:45:21.165 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.nK2jDAtgu7 is 9KB 2023-09-30T00:45:21.179 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.nK2jDAtgu7 2023-09-30T00:45:21.186 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-09-30T00:45:21.241 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:45:21.241 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-09-30T00:45:21.241 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-09-30T00:45:21.280 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-163-generic 2023-09-30T00:45:21.280 DEBUG:teuthology.orchestra.run.smithi069:> sudo update-grub 2023-09-30T00:45:21.351 INFO:teuthology.orchestra.run.smithi150.stdout:linux-image-generic is already the newest version (5.4.0.163.160). 2023-09-30T00:45:21.351 INFO:teuthology.orchestra.run.smithi150.stdout:0 upgraded, 0 newly installed, 0 to remove and 263 not upgraded. 2023-09-30T00:45:21.353 DEBUG:teuthology.orchestra.run.smithi150:> dpkg -s linux-image-generic 2023-09-30T00:45:21.374 INFO:teuthology.orchestra.run.smithi150.stdout:Package: linux-image-generic 2023-09-30T00:45:21.374 INFO:teuthology.orchestra.run.smithi150.stdout:Status: install ok installed 2023-09-30T00:45:21.374 INFO:teuthology.orchestra.run.smithi150.stdout:Priority: optional 2023-09-30T00:45:21.374 INFO:teuthology.orchestra.run.smithi150.stdout:Section: kernel 2023-09-30T00:45:21.375 INFO:teuthology.orchestra.run.smithi150.stdout:Installed-Size: 20 2023-09-30T00:45:21.375 INFO:teuthology.orchestra.run.smithi150.stdout:Maintainer: Ubuntu Kernel Team 2023-09-30T00:45:21.375 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: amd64 2023-09-30T00:45:21.375 INFO:teuthology.orchestra.run.smithi150.stdout:Source: linux-meta 2023-09-30T00:45:21.375 INFO:teuthology.orchestra.run.smithi150.stdout:Version: 5.4.0.163.160 2023-09-30T00:45:21.375 INFO:teuthology.orchestra.run.smithi150.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-30T00:45:21.375 INFO:teuthology.orchestra.run.smithi150.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-30T00:45:21.375 INFO:teuthology.orchestra.run.smithi150.stdout:Recommends: thermald 2023-09-30T00:45:21.375 INFO:teuthology.orchestra.run.smithi150.stdout:Description: Generic Linux kernel image 2023-09-30T00:45:21.375 INFO:teuthology.orchestra.run.smithi150.stdout: This package will always depend on the latest generic kernel image 2023-09-30T00:45:21.375 INFO:teuthology.orchestra.run.smithi150.stdout: available. 2023-09-30T00:45:21.376 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2023-09-30T00:45:21.380 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.BgqXZ53dRa 2023-09-30T00:45:21.380 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub/grub.cfg /tmp/tmp.BgqXZ53dRa 2023-09-30T00:45:21.437 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.BgqXZ53dRa 2023-09-30T00:45:21.467 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.BgqXZ53dRa is 9KB 2023-09-30T00:45:21.480 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.BgqXZ53dRa 2023-09-30T00:45:21.486 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-09-30T00:45:21.544 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:45:21.545 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-09-30T00:45:21.545 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-09-30T00:45:21.578 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-163-generic 2023-09-30T00:45:21.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo update-grub 2023-09-30T00:45:22.591 INFO:teuthology.orchestra.run.smithi069.stderr:Sourcing file `/etc/default/grub' 2023-09-30T00:45:22.597 INFO:teuthology.orchestra.run.smithi069.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-30T00:45:22.607 INFO:teuthology.orchestra.run.smithi069.stderr:Generating grub configuration file ... 2023-09-30T00:45:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:45:22.864 INFO:teuthology.orchestra.run.smithi150.stderr:Sourcing file `/etc/default/grub' 2023-09-30T00:45:22.871 INFO:teuthology.orchestra.run.smithi150.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-30T00:45:22.872 INFO:teuthology.orchestra.run.smithi069.stderr:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:45:22.885 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2023-09-30T00:45:23.141 INFO:teuthology.orchestra.run.smithi150.stderr:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-30T00:45:23.153 INFO:teuthology.orchestra.run.smithi150.stderr:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-30T00:45:23.275 INFO:teuthology.orchestra.run.smithi069.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-30T00:45:23.286 INFO:teuthology.orchestra.run.smithi069.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-30T00:45:23.508 INFO:teuthology.orchestra.run.smithi150.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-30T00:45:23.519 INFO:teuthology.orchestra.run.smithi150.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-30T00:45:24.527 INFO:teuthology.orchestra.run.smithi069.stderr:done 2023-09-30T00:45:24.528 DEBUG:teuthology.orchestra.run.smithi069:> sudo shutdown -r now 2023-09-30T00:45:25.219 INFO:teuthology.orchestra.run.smithi150.stderr:done 2023-09-30T00:45:25.221 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2023-09-30T00:45:54.561 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-09-30T00:45:54.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-09-30T00:45:54.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:45:55.224 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-09-30T00:45:55.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-09-30T00:45:55.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:46:29.242 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-09-30T00:46:29.754 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-09-30T00:46:38.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-09-30T00:46:38.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:46:38.756 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-09-30T00:46:38.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:46:56.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-09-30T00:46:57.146 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-09-30T00:47:08.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-09-30T00:47:08.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:47:08.882 DEBUG:teuthology.orchestra.run.smithi069:> true 2023-09-30T00:47:09.147 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-09-30T00:47:09.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:47:09.481 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-09-30T00:47:09.481 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-163-generic"... 2023-09-30T00:47:09.482 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2023-09-30T00:47:09.488 INFO:teuthology.orchestra.run.smithi069.stdout:5.4.0-163-generic 2023-09-30T00:47:09.488 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-163-generic vs 5.4.0-163-generic 2023-09-30T00:47:09.488 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-30T00:47:09.488 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-09-30T00:47:09.523 DEBUG:teuthology.orchestra.run.smithi150:> true 2023-09-30T00:47:10.444 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-09-30T00:47:10.444 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-163-generic"... 2023-09-30T00:47:10.445 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2023-09-30T00:47:10.451 INFO:teuthology.orchestra.run.smithi150.stdout:5.4.0-163-generic 2023-09-30T00:47:10.451 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-163-generic vs 5.4.0-163-generic 2023-09-30T00:47:10.451 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-30T00:47:10.451 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-09-30T00:47:10.489 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-09-30T00:47:10.490 DEBUG:teuthology.orchestra.run.smithi069:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-30T00:47:10.569 INFO:teuthology.orchestra.run.smithi069.stdout:ttyS1 2023-09-30T00:47:10.591 DEBUG:teuthology.parallel:result is None 2023-09-30T00:47:11.451 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-09-30T00:47:11.452 DEBUG:teuthology.orchestra.run.smithi150:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-30T00:47:11.863 INFO:teuthology.orchestra.run.smithi150.stdout:ttyS1 2023-09-30T00:47:11.884 DEBUG:teuthology.parallel:result is None 2023-09-30T00:47:11.885 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-30T00:47:11.892 INFO:teuthology.task.internal:Creating test directory... 2023-09-30T00:47:11.892 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T00:47:11.895 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T00:47:11.901 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-30T00:47:11.929 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-30T00:47:11.962 INFO:teuthology.task.internal:Creating archive directory... 2023-09-30T00:47:11.962 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T00:47:11.965 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T00:47:12.005 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-30T00:47:12.012 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-30T00:47:12.012 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T00:47:12.022 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T00:47:12.037 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T00:47:12.046 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T00:47:12.071 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T00:47:12.080 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T00:47:12.082 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-30T00:47:12.089 INFO:teuthology.task.internal:Configuring sudo... 2023-09-30T00:47:12.089 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T00:47:12.092 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T00:47:12.107 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-30T00:47:12.156 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-30T00:47:12.157 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T00:47:12.159 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T00:47:12.166 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T00:47:12.173 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T00:47:12.223 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:47:12.223 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T00:47:12.245 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T00:47:12.251 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T00:47:12.258 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:47:12.258 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T00:47:12.280 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2023-09-30T00:47:12.283 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2023-09-30T00:47:12.363 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-30T00:47:12.370 INFO:teuthology.task.internal:Starting timer... 2023-09-30T00:47:12.370 INFO:teuthology.run_tasks:Running task pcp... 2023-09-30T00:47:12.408 INFO:teuthology.run_tasks:Running task selinux... 2023-09-30T00:47:12.439 DEBUG:teuthology.task.selinux:Excluding smithi069: OS 'ubuntu' does not support SELinux 2023-09-30T00:47:12.439 DEBUG:teuthology.task.selinux:Excluding smithi150: OS 'ubuntu' does not support SELinux 2023-09-30T00:47:12.440 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-30T00:47:12.440 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-09-30T00:47:12.440 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-30T00:47:12.440 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-30T00:47:12.473 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-30T00:47:12.479 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-30T00:47:12.589 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-30T00:47:12.615 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-30T00:47:12.616 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi069.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-30T00:52:52.736 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2023-09-30T00:52:52.737 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-09-30T00:52:52.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:52:52.788 DEBUG:teuthology.orchestra.run.smithi069:> true 2023-09-30T00:52:52.911 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-09-30T00:52:52.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-09-30T00:52:52.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T00:52:52.960 DEBUG:teuthology.orchestra.run.smithi150:> true 2023-09-30T00:52:53.079 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-09-30T00:52:53.080 INFO:teuthology.run_tasks:Running task clock... 2023-09-30T00:52:53.094 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-30T00:52:53.094 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T00:52:53.095 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T00:52:53.097 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T00:52:53.097 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T00:52:53.127 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-09-30T00:52:53.128 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: Command line: ntpd -gq 2023-09-30T00:52:53.128 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-09-30T00:52:53.128 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: Command line: ntpd -gq 2023-09-30T00:52:53.128 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: proto: precision = 0.100 usec (-23) 2023-09-30T00:52:53.129 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: proto: precision = 0.232 usec (-22) 2023-09-30T00:52:53.129 INFO:teuthology.orchestra.run.smithi150.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-30T00:52:53.130 INFO:teuthology.orchestra.run.smithi069.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-30T00:52:53.130 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-30T00:52:53.130 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-30T00:52:53.131 INFO:teuthology.orchestra.run.smithi150.stderr:restrict ::: KOD does nothing without LIMITED. 2023-09-30T00:52:53.131 INFO:teuthology.orchestra.run.smithi069.stderr:restrict ::: KOD does nothing without LIMITED. 2023-09-30T00:52:53.132 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: restrict ::: KOD does nothing without LIMITED. 2023-09-30T00:52:53.133 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: Listen and drop on 0 v6wildcard [::]:123 2023-09-30T00:52:53.133 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-09-30T00:52:53.133 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: Listen normally on 2 lo 127.0.0.1:123 2023-09-30T00:52:53.133 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: Listen normally on 3 enp3s0f1 172.21.15.150:123 2023-09-30T00:52:53.133 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: Listen normally on 4 lo [::1]:123 2023-09-30T00:52:53.133 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e35%5]:123 2023-09-30T00:52:53.133 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:52:53 ntpd[22709]: Listening on routing socket on fd #22 for interface updates 2023-09-30T00:52:53.133 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: restrict ::: KOD does nothing without LIMITED. 2023-09-30T00:52:53.134 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: Listen and drop on 0 v6wildcard [::]:123 2023-09-30T00:52:53.134 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-09-30T00:52:53.134 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: Listen normally on 2 lo 127.0.0.1:123 2023-09-30T00:52:53.134 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: Listen normally on 3 enp3s0f1 172.21.15.69:123 2023-09-30T00:52:53.134 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: Listen normally on 4 lo [::1]:123 2023-09-30T00:52:53.134 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d737%5]:123 2023-09-30T00:52:53.134 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:52:53 ntpd[22707]: Listening on routing socket on fd #22 for interface updates 2023-09-30T00:52:54.128 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:54 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:54.128 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:54 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:54.128 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:54 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:54.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:54 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:54.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:54 ntpd[22707]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:54.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:54 ntpd[22707]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:54.129 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:54 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:54.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:54 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:54.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:54 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:54.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:54 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:54.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:54 ntpd[22709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:54.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:54 ntpd[22709]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:55.128 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:55 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:55.128 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:55 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:55.128 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:55 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:55.128 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:55 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:55.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:55 ntpd[22707]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:55.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:55 ntpd[22707]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:55.129 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:55 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:55.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:55 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:55.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:55 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:55.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:55 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:55.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:55 ntpd[22709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:55.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:55 ntpd[22709]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:56.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:56.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:56.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:56.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:56.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:56.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:56.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:56.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:56.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:56.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:56.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:56.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:56 ntpd[22707]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:56.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:56 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:56.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:56 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:56.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:56 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:56.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:56 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:56.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:56 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:56.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:56 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:56.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:56 ntpd[22709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:56.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:56 ntpd[22709]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:57.128 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:57 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:57.128 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:57 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:57.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:57 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:57.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:57 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:57.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:57 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:57.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:57 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:57.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:57 ntpd[22707]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:57.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:57 ntpd[22707]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:57.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:57 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:57.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:57 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:57.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:57 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:57.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:57 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:57.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:57 ntpd[22709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:57.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:57 ntpd[22709]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:58.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:58.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:58.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:58.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:58.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:58.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:58.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:58.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:58.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:58.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:58.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:58.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:58 ntpd[22707]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:58.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:58.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:58.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:58.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:58.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:58.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:58.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:58.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:58.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:58.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:58.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:58.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:58 ntpd[22709]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:59 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:59 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:59.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:59 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:59.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:59 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:59.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:59 ntpd[22707]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:59.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:52:59 ntpd[22707]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:52:59.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:59 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:59.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:59 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:59.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:59 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:52:59.130 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:59 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:52:59.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:59 ntpd[22709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:52:59.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:52:59 ntpd[22709]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:53:00.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:53:00 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:53:00.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:53:00 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:53:00.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:53:00 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:53:00.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:53:00 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:53:00.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:53:00 ntpd[22707]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:53:00.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:53:00 ntpd[22707]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:53:00.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:53:00 ntpd[22707]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:53:00.129 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:53:00 ntpd[22707]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:53:00.130 INFO:teuthology.orchestra.run.smithi069.stdout:30 Sep 00:53:00 ntpd[22707]: ntpd: time slew -0.000910 s 2023-09-30T00:53:00.130 INFO:teuthology.orchestra.run.smithi069.stdout:ntpd: time slew -0.000910s 2023-09-30T00:53:00.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:53:00 ntpd[22707]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-09-30T00:53:00.130 INFO:teuthology.orchestra.run.smithi069.stderr:30 Sep 00:53:00 ntpd[22707]: can't open /var/log/ntpstats/loopstats.20230930: Permission denied 2023-09-30T00:53:00.131 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:53:00 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:53:00.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:53:00 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:53:00.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:53:00 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:53:00.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:53:00 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:53:00.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:53:00 ntpd[22709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-30T00:53:00.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:53:00 ntpd[22709]: can't open /var/log/ntpstats/rawstats.20230930: Permission denied 2023-09-30T00:53:00.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:53:00 ntpd[22709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-30T00:53:00.132 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:53:00 ntpd[22709]: can't open /var/log/ntpstats/peerstats.20230930: Permission denied 2023-09-30T00:53:00.132 INFO:teuthology.orchestra.run.smithi150.stdout:30 Sep 00:53:00 ntpd[22709]: ntpd: time slew -0.000681 s 2023-09-30T00:53:00.133 INFO:teuthology.orchestra.run.smithi150.stdout:ntpd: time slew -0.000681s 2023-09-30T00:53:00.133 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:53:00 ntpd[22709]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-09-30T00:53:00.133 INFO:teuthology.orchestra.run.smithi150.stderr:30 Sep 00:53:00 ntpd[22709]: can't open /var/log/ntpstats/loopstats.20230930: Permission denied 2023-09-30T00:53:00.188 INFO:teuthology.orchestra.run.smithi150.stdout: remote refid st t when poll reach delay offset jitter 2023-09-30T00:53:00.188 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================== 2023-09-30T00:53:00.188 INFO:teuthology.orchestra.run.smithi150.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:53:00.189 INFO:teuthology.orchestra.run.smithi150.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:53:00.189 INFO:teuthology.orchestra.run.smithi150.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:53:00.189 INFO:teuthology.orchestra.run.smithi150.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:53:00.190 INFO:teuthology.orchestra.run.smithi069.stdout: remote refid st t when poll reach delay offset jitter 2023-09-30T00:53:00.190 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================== 2023-09-30T00:53:00.190 INFO:teuthology.orchestra.run.smithi069.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:53:00.190 INFO:teuthology.orchestra.run.smithi069.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:53:00.190 INFO:teuthology.orchestra.run.smithi069.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:53:00.191 INFO:teuthology.orchestra.run.smithi069.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T00:53:00.192 INFO:teuthology.run_tasks:Running task install... 2023-09-30T00:53:00.202 DEBUG:teuthology.task.install:project ceph 2023-09-30T00:53:00.202 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}} 2023-09-30T00:53:00.203 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'} 2023-09-30T00:53:00.203 INFO:teuthology.task.install:Using flavor: default 2023-09-30T00:53:00.209 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-30T00:53:00.209 INFO:teuthology.task.install:extra packages: [] 2023-09-30T00:53:00.210 DEBUG:teuthology.orchestra.run.smithi069:> sudo apt-key list | grep Ceph 2023-09-30T00:53:00.210 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-key list | grep Ceph 2023-09-30T00:53:00.309 INFO:teuthology.orchestra.run.smithi069.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-09-30T00:53:00.333 INFO:teuthology.orchestra.run.smithi150.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-09-30T00:53:00.369 INFO:teuthology.orchestra.run.smithi069.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-09-30T00:53:00.369 INFO:teuthology.orchestra.run.smithi069.stdout:uid [ unknown] Ceph.com (release key) 2023-09-30T00:53:00.370 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-09-30T00:53:00.371 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-30T00:53:00.392 INFO:teuthology.orchestra.run.smithi150.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-09-30T00:53:00.392 INFO:teuthology.orchestra.run.smithi150.stdout:uid [ unknown] Ceph.com (release key) 2023-09-30T00:53:00.393 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-09-30T00:53:00.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-30T00:53:00.521 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default/ 2023-09-30T00:53:00.522 INFO:teuthology.task.install.deb:Package version is 17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:53:00.536 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default/ 2023-09-30T00:53:00.536 INFO:teuthology.task.install.deb:Package version is 17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:53:00.644 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:53:00.644 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-09-30T00:53:00.661 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2023-09-30T00:53:00.699 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:53:00.699 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-09-30T00:53:00.716 DEBUG:teuthology.orchestra.run.smithi069:> sudo apt-get update 2023-09-30T00:53:00.805 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:53:00.813 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-30T00:53:00.833 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-30T00:53:00.895 INFO:teuthology.orchestra.run.smithi069.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-30T00:53:00.895 INFO:teuthology.orchestra.run.smithi069.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-30T00:53:00.901 INFO:teuthology.orchestra.run.smithi069.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-30T00:53:00.917 INFO:teuthology.orchestra.run.smithi150.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal InRelease 2023-09-30T00:53:00.917 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-30T00:53:00.918 INFO:teuthology.orchestra.run.smithi069.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-30T00:53:00.950 INFO:teuthology.orchestra.run.smithi069.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal InRelease 2023-09-30T00:53:00.954 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-09-30T00:53:00.971 INFO:teuthology.orchestra.run.smithi069.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-09-30T00:53:00.991 INFO:teuthology.orchestra.run.smithi150.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release.gpg 2023-09-30T00:53:00.994 INFO:teuthology.orchestra.run.smithi069.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release.gpg 2023-09-30T00:53:01.016 INFO:teuthology.orchestra.run.smithi069.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-09-30T00:53:01.028 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-09-30T00:53:01.051 INFO:teuthology.orchestra.run.smithi069.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main i386 Packages [3,375 B] 2023-09-30T00:53:01.087 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main i386 Packages [3,375 B] 2023-09-30T00:53:01.446 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 28.4 kB in 1s (45.4 kB/s) 2023-09-30T00:53:01.547 INFO:teuthology.orchestra.run.smithi069.stdout:Fetched 28.4 kB in 1s (45.1 kB/s) 2023-09-30T00:53:02.470 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-09-30T00:53:02.494 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1165-g93dce5e1-1focal cephadm=17.2.6-1165-g93dce5e1-1focal ceph-mds=17.2.6-1165-g93dce5e1-1focal ceph-mgr=17.2.6-1165-g93dce5e1-1focal ceph-common=17.2.6-1165-g93dce5e1-1focal ceph-fuse=17.2.6-1165-g93dce5e1-1focal ceph-test=17.2.6-1165-g93dce5e1-1focal ceph-volume=17.2.6-1165-g93dce5e1-1focal radosgw=17.2.6-1165-g93dce5e1-1focal python3-rados=17.2.6-1165-g93dce5e1-1focal python3-rgw=17.2.6-1165-g93dce5e1-1focal python3-cephfs=17.2.6-1165-g93dce5e1-1focal python3-rbd=17.2.6-1165-g93dce5e1-1focal libcephfs2=17.2.6-1165-g93dce5e1-1focal libcephfs-dev=17.2.6-1165-g93dce5e1-1focal librados2=17.2.6-1165-g93dce5e1-1focal librbd1=17.2.6-1165-g93dce5e1-1focal rbd-fuse=17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:53:02.562 INFO:teuthology.orchestra.run.smithi069.stdout:Reading package lists... 2023-09-30T00:53:02.565 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-09-30T00:53:02.587 DEBUG:teuthology.orchestra.run.smithi069:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1165-g93dce5e1-1focal cephadm=17.2.6-1165-g93dce5e1-1focal ceph-mds=17.2.6-1165-g93dce5e1-1focal ceph-mgr=17.2.6-1165-g93dce5e1-1focal ceph-common=17.2.6-1165-g93dce5e1-1focal ceph-fuse=17.2.6-1165-g93dce5e1-1focal ceph-test=17.2.6-1165-g93dce5e1-1focal ceph-volume=17.2.6-1165-g93dce5e1-1focal radosgw=17.2.6-1165-g93dce5e1-1focal python3-rados=17.2.6-1165-g93dce5e1-1focal python3-rgw=17.2.6-1165-g93dce5e1-1focal python3-cephfs=17.2.6-1165-g93dce5e1-1focal python3-rbd=17.2.6-1165-g93dce5e1-1focal libcephfs2=17.2.6-1165-g93dce5e1-1focal libcephfs-dev=17.2.6-1165-g93dce5e1-1focal librados2=17.2.6-1165-g93dce5e1-1focal librbd1=17.2.6-1165-g93dce5e1-1focal rbd-fuse=17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:53:02.656 INFO:teuthology.orchestra.run.smithi069.stdout:Reading package lists... 2023-09-30T00:53:02.764 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2023-09-30T00:53:02.764 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2023-09-30T00:53:02.856 INFO:teuthology.orchestra.run.smithi069.stdout:Building dependency tree... 2023-09-30T00:53:02.856 INFO:teuthology.orchestra.run.smithi069.stdout:Reading state information... 2023-09-30T00:53:02.933 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages were automatically installed and are no longer required: 2023-09-30T00:53:02.934 INFO:teuthology.orchestra.run.smithi150.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-09-30T00:53:02.934 INFO:teuthology.orchestra.run.smithi150.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-30T00:53:02.934 INFO:teuthology.orchestra.run.smithi150.stdout:The following additional packages will be installed: 2023-09-30T00:53:02.935 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-09-30T00:53:02.935 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-09-30T00:53:02.935 INFO:teuthology.orchestra.run.smithi150.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-09-30T00:53:02.935 INFO:teuthology.orchestra.run.smithi150.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-09-30T00:53:02.935 INFO:teuthology.orchestra.run.smithi150.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-09-30T00:53:02.937 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-09-30T00:53:02.937 INFO:teuthology.orchestra.run.smithi150.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-09-30T00:53:02.937 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-09-30T00:53:02.937 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2023-09-30T00:53:02.937 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-09-30T00:53:02.938 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-09-30T00:53:02.938 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-09-30T00:53:02.938 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-09-30T00:53:02.938 INFO:teuthology.orchestra.run.smithi150.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-09-30T00:53:02.938 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-09-30T00:53:02.939 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2023-09-30T00:53:02.939 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2023-09-30T00:53:02.939 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-09-30T00:53:02.939 INFO:teuthology.orchestra.run.smithi150.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2023-09-30T00:53:02.939 INFO:teuthology.orchestra.run.smithi150.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2023-09-30T00:53:02.939 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2023-09-30T00:53:02.940 INFO:teuthology.orchestra.run.smithi150.stdout: socat xmlstarlet 2023-09-30T00:53:02.940 INFO:teuthology.orchestra.run.smithi150.stdout:Suggested packages: 2023-09-30T00:53:02.940 INFO:teuthology.orchestra.run.smithi150.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-09-30T00:53:02.940 INFO:teuthology.orchestra.run.smithi150.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-09-30T00:53:02.940 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-09-30T00:53:02.940 INFO:teuthology.orchestra.run.smithi150.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-09-30T00:53:02.941 INFO:teuthology.orchestra.run.smithi150.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-09-30T00:53:02.941 INFO:teuthology.orchestra.run.smithi150.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-09-30T00:53:02.941 INFO:teuthology.orchestra.run.smithi150.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-09-30T00:53:02.941 INFO:teuthology.orchestra.run.smithi150.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-09-30T00:53:02.941 INFO:teuthology.orchestra.run.smithi150.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-09-30T00:53:02.941 INFO:teuthology.orchestra.run.smithi150.stdout:Recommended packages: 2023-09-30T00:53:02.942 INFO:teuthology.orchestra.run.smithi150.stdout: btrfs-tools 2023-09-30T00:53:03.026 INFO:teuthology.orchestra.run.smithi069.stdout:The following packages were automatically installed and are no longer required: 2023-09-30T00:53:03.027 INFO:teuthology.orchestra.run.smithi069.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-09-30T00:53:03.027 INFO:teuthology.orchestra.run.smithi069.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-30T00:53:03.027 INFO:teuthology.orchestra.run.smithi069.stdout:The following additional packages will be installed: 2023-09-30T00:53:03.028 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-09-30T00:53:03.028 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-09-30T00:53:03.028 INFO:teuthology.orchestra.run.smithi069.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-09-30T00:53:03.028 INFO:teuthology.orchestra.run.smithi069.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-09-30T00:53:03.028 INFO:teuthology.orchestra.run.smithi069.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-09-30T00:53:03.029 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-09-30T00:53:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-09-30T00:53:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-09-30T00:53:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2023-09-30T00:53:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-09-30T00:53:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-09-30T00:53:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-09-30T00:53:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-09-30T00:53:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-09-30T00:53:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-09-30T00:53:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2023-09-30T00:53:03.030 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout: socat xmlstarlet 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout:Suggested packages: 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-09-30T00:53:03.031 INFO:teuthology.orchestra.run.smithi069.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-09-30T00:53:03.032 INFO:teuthology.orchestra.run.smithi069.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-09-30T00:53:03.032 INFO:teuthology.orchestra.run.smithi069.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-09-30T00:53:03.032 INFO:teuthology.orchestra.run.smithi069.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-09-30T00:53:03.032 INFO:teuthology.orchestra.run.smithi069.stdout:Recommended packages: 2023-09-30T00:53:03.032 INFO:teuthology.orchestra.run.smithi069.stdout: btrfs-tools 2023-09-30T00:53:03.051 INFO:teuthology.orchestra.run.smithi150.stdout:The following NEW packages will be installed: 2023-09-30T00:53:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-09-30T00:53:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-09-30T00:53:03.051 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-09-30T00:53:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2023-09-30T00:53:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2023-09-30T00:53:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2023-09-30T00:53:03.052 INFO:teuthology.orchestra.run.smithi150.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2023-09-30T00:53:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2023-09-30T00:53:03.053 INFO:teuthology.orchestra.run.smithi150.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2023-09-30T00:53:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-09-30T00:53:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2023-09-30T00:53:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-09-30T00:53:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-09-30T00:53:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2023-09-30T00:53:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2023-09-30T00:53:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2023-09-30T00:53:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-09-30T00:53:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2023-09-30T00:53:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2023-09-30T00:53:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-09-30T00:53:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-09-30T00:53:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-09-30T00:53:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-09-30T00:53:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-09-30T00:53:03.055 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages will be upgraded: 2023-09-30T00:53:03.055 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 librbd1 2023-09-30T00:53:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:2 upgraded, 108 newly installed, 0 to remove and 245 not upgraded. 2023-09-30T00:53:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:Need to get 209 MB of archives. 2023-09-30T00:53:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:After this operation, 980 MB of additional disk space will be used. 2023-09-30T00:53:03.138 INFO:teuthology.orchestra.run.smithi150.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-09-30T00:53:03.143 INFO:teuthology.orchestra.run.smithi069.stdout:The following NEW packages will be installed: 2023-09-30T00:53:03.143 INFO:teuthology.orchestra.run.smithi069.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-09-30T00:53:03.143 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-09-30T00:53:03.143 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-09-30T00:53:03.145 INFO:teuthology.orchestra.run.smithi069.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2023-09-30T00:53:03.145 INFO:teuthology.orchestra.run.smithi069.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2023-09-30T00:53:03.145 INFO:teuthology.orchestra.run.smithi069.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2023-09-30T00:53:03.146 INFO:teuthology.orchestra.run.smithi069.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2023-09-30T00:53:03.146 INFO:teuthology.orchestra.run.smithi069.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2023-09-30T00:53:03.146 INFO:teuthology.orchestra.run.smithi069.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2023-09-30T00:53:03.146 INFO:teuthology.orchestra.run.smithi069.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-09-30T00:53:03.146 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2023-09-30T00:53:03.146 INFO:teuthology.orchestra.run.smithi069.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-09-30T00:53:03.146 INFO:teuthology.orchestra.run.smithi069.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-09-30T00:53:03.146 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2023-09-30T00:53:03.146 INFO:teuthology.orchestra.run.smithi069.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2023-09-30T00:53:03.146 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2023-09-30T00:53:03.146 INFO:teuthology.orchestra.run.smithi069.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-09-30T00:53:03.147 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2023-09-30T00:53:03.147 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2023-09-30T00:53:03.147 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-09-30T00:53:03.147 INFO:teuthology.orchestra.run.smithi069.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-09-30T00:53:03.147 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-09-30T00:53:03.147 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-09-30T00:53:03.147 INFO:teuthology.orchestra.run.smithi069.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-09-30T00:53:03.147 INFO:teuthology.orchestra.run.smithi069.stdout:The following packages will be upgraded: 2023-09-30T00:53:03.148 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 librbd1 2023-09-30T00:53:03.202 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1165-g93dce5e1-1focal [3,501 kB] 2023-09-30T00:53:03.238 INFO:teuthology.orchestra.run.smithi069.stdout:2 upgraded, 108 newly installed, 0 to remove and 245 not upgraded. 2023-09-30T00:53:03.238 INFO:teuthology.orchestra.run.smithi069.stdout:Need to get 209 MB of archives. 2023-09-30T00:53:03.238 INFO:teuthology.orchestra.run.smithi069.stdout:After this operation, 980 MB of additional disk space will be used. 2023-09-30T00:53:03.239 INFO:teuthology.orchestra.run.smithi069.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-09-30T00:53:03.285 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-09-30T00:53:03.287 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-09-30T00:53:03.294 INFO:teuthology.orchestra.run.smithi069.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1165-g93dce5e1-1focal [3,501 kB] 2023-09-30T00:53:03.305 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-09-30T00:53:03.320 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-09-30T00:53:03.376 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-09-30T00:53:03.379 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-09-30T00:53:03.389 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-09-30T00:53:03.410 INFO:teuthology.orchestra.run.smithi069.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-09-30T00:53:03.411 INFO:teuthology.orchestra.run.smithi069.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-09-30T00:53:03.416 INFO:teuthology.orchestra.run.smithi150.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1165-g93dce5e1-1focal [3,552 kB] 2023-09-30T00:53:03.433 INFO:teuthology.orchestra.run.smithi069.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-09-30T00:53:03.450 INFO:teuthology.orchestra.run.smithi069.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-09-30T00:53:03.463 INFO:teuthology.orchestra.run.smithi150.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-09-30T00:53:03.468 INFO:teuthology.orchestra.run.smithi150.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-09-30T00:53:03.481 INFO:teuthology.orchestra.run.smithi150.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-09-30T00:53:03.485 INFO:teuthology.orchestra.run.smithi150.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-09-30T00:53:03.486 INFO:teuthology.orchestra.run.smithi150.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-09-30T00:53:03.490 INFO:teuthology.orchestra.run.smithi150.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-09-30T00:53:03.491 INFO:teuthology.orchestra.run.smithi150.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-09-30T00:53:03.491 INFO:teuthology.orchestra.run.smithi150.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-09-30T00:53:03.494 INFO:teuthology.orchestra.run.smithi150.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-09-30T00:53:03.506 INFO:teuthology.orchestra.run.smithi150.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-09-30T00:53:03.508 INFO:teuthology.orchestra.run.smithi069.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-09-30T00:53:03.512 INFO:teuthology.orchestra.run.smithi069.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-09-30T00:53:03.530 INFO:teuthology.orchestra.run.smithi150.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-09-30T00:53:03.531 INFO:teuthology.orchestra.run.smithi069.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-09-30T00:53:03.531 INFO:teuthology.orchestra.run.smithi150.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-09-30T00:53:03.531 INFO:teuthology.orchestra.run.smithi150.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-09-30T00:53:03.533 INFO:teuthology.orchestra.run.smithi150.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-09-30T00:53:03.534 INFO:teuthology.orchestra.run.smithi150.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-09-30T00:53:03.534 INFO:teuthology.orchestra.run.smithi150.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-09-30T00:53:03.538 INFO:teuthology.orchestra.run.smithi150.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-09-30T00:53:03.538 INFO:teuthology.orchestra.run.smithi150.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-09-30T00:53:03.540 INFO:teuthology.orchestra.run.smithi150.stdout:Get:29 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1165-g93dce5e1-1focal [758 kB] 2023-09-30T00:53:03.542 INFO:teuthology.orchestra.run.smithi150.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-09-30T00:53:03.569 INFO:teuthology.orchestra.run.smithi150.stdout:Get:31 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1165-g93dce5e1-1focal [414 kB] 2023-09-30T00:53:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-09-30T00:53:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-09-30T00:53:03.580 INFO:teuthology.orchestra.run.smithi150.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-09-30T00:53:03.581 INFO:teuthology.orchestra.run.smithi150.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-09-30T00:53:03.582 INFO:teuthology.orchestra.run.smithi150.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-09-30T00:53:03.582 INFO:teuthology.orchestra.run.smithi150.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-09-30T00:53:03.582 INFO:teuthology.orchestra.run.smithi150.stdout:Get:38 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1165-g93dce5e1-1focal [22.5 kB] 2023-09-30T00:53:03.584 INFO:teuthology.orchestra.run.smithi150.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-09-30T00:53:03.586 INFO:teuthology.orchestra.run.smithi150.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-09-30T00:53:03.587 INFO:teuthology.orchestra.run.smithi150.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-09-30T00:53:03.588 INFO:teuthology.orchestra.run.smithi150.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1165-g93dce5e1-1focal [200 kB] 2023-09-30T00:53:03.595 INFO:teuthology.orchestra.run.smithi150.stdout:Get:43 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1165-g93dce5e1-1focal [47.1 kB] 2023-09-30T00:53:03.596 INFO:teuthology.orchestra.run.smithi150.stdout:Get:44 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1165-g93dce5e1-1focal [385 kB] 2023-09-30T00:53:03.609 INFO:teuthology.orchestra.run.smithi150.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1165-g93dce5e1-1focal [5,569 kB] 2023-09-30T00:53:03.624 INFO:teuthology.orchestra.run.smithi150.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-09-30T00:53:03.626 INFO:teuthology.orchestra.run.smithi150.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-09-30T00:53:03.628 INFO:teuthology.orchestra.run.smithi150.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-09-30T00:53:03.630 INFO:teuthology.orchestra.run.smithi150.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-09-30T00:53:03.646 INFO:teuthology.orchestra.run.smithi069.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-09-30T00:53:03.648 INFO:teuthology.orchestra.run.smithi069.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-09-30T00:53:03.654 INFO:teuthology.orchestra.run.smithi150.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-09-30T00:53:03.655 INFO:teuthology.orchestra.run.smithi150.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-09-30T00:53:03.658 INFO:teuthology.orchestra.run.smithi069.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-09-30T00:53:03.660 INFO:teuthology.orchestra.run.smithi069.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-09-30T00:53:03.660 INFO:teuthology.orchestra.run.smithi069.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-09-30T00:53:03.662 INFO:teuthology.orchestra.run.smithi069.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-09-30T00:53:03.754 INFO:teuthology.orchestra.run.smithi069.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-09-30T00:53:03.754 INFO:teuthology.orchestra.run.smithi069.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-09-30T00:53:03.757 INFO:teuthology.orchestra.run.smithi069.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-09-30T00:53:03.758 INFO:teuthology.orchestra.run.smithi069.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-09-30T00:53:03.759 INFO:teuthology.orchestra.run.smithi069.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-09-30T00:53:03.805 INFO:teuthology.orchestra.run.smithi069.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-09-30T00:53:03.805 INFO:teuthology.orchestra.run.smithi069.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-09-30T00:53:03.806 INFO:teuthology.orchestra.run.smithi069.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-09-30T00:53:03.807 INFO:teuthology.orchestra.run.smithi069.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-09-30T00:53:03.824 INFO:teuthology.orchestra.run.smithi150.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1165-g93dce5e1-1focal [116 kB] 2023-09-30T00:53:03.826 INFO:teuthology.orchestra.run.smithi150.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1165-g93dce5e1-1focal [480 kB] 2023-09-30T00:53:03.856 INFO:teuthology.orchestra.run.smithi069.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-09-30T00:53:03.858 INFO:teuthology.orchestra.run.smithi069.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-09-30T00:53:03.859 INFO:teuthology.orchestra.run.smithi069.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-09-30T00:53:03.860 INFO:teuthology.orchestra.run.smithi069.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-09-30T00:53:03.862 INFO:teuthology.orchestra.run.smithi069.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-09-30T00:53:03.863 INFO:teuthology.orchestra.run.smithi069.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-09-30T00:53:03.869 INFO:teuthology.orchestra.run.smithi069.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-09-30T00:53:03.870 INFO:teuthology.orchestra.run.smithi069.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-09-30T00:53:03.870 INFO:teuthology.orchestra.run.smithi069.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-09-30T00:53:03.899 INFO:teuthology.orchestra.run.smithi069.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-09-30T00:53:03.911 INFO:teuthology.orchestra.run.smithi069.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-09-30T00:53:03.913 INFO:teuthology.orchestra.run.smithi150.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1165-g93dce5e1-1focal [27.7 MB] 2023-09-30T00:53:03.913 INFO:teuthology.orchestra.run.smithi150.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-09-30T00:53:03.916 INFO:teuthology.orchestra.run.smithi069.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-09-30T00:53:03.918 INFO:teuthology.orchestra.run.smithi069.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-09-30T00:53:03.918 INFO:teuthology.orchestra.run.smithi069.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-09-30T00:53:03.920 INFO:teuthology.orchestra.run.smithi069.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-09-30T00:53:03.921 INFO:teuthology.orchestra.run.smithi069.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-09-30T00:53:03.922 INFO:teuthology.orchestra.run.smithi069.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-09-30T00:53:03.922 INFO:teuthology.orchestra.run.smithi069.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-09-30T00:53:03.923 INFO:teuthology.orchestra.run.smithi069.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-09-30T00:53:03.937 INFO:teuthology.orchestra.run.smithi150.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-09-30T00:53:03.939 INFO:teuthology.orchestra.run.smithi150.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-09-30T00:53:03.971 INFO:teuthology.orchestra.run.smithi150.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-09-30T00:53:03.971 INFO:teuthology.orchestra.run.smithi150.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-09-30T00:53:03.972 INFO:teuthology.orchestra.run.smithi150.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-09-30T00:53:03.972 INFO:teuthology.orchestra.run.smithi150.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-09-30T00:53:03.973 INFO:teuthology.orchestra.run.smithi150.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-09-30T00:53:03.974 INFO:teuthology.orchestra.run.smithi150.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-09-30T00:53:03.974 INFO:teuthology.orchestra.run.smithi150.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-09-30T00:53:03.985 INFO:teuthology.orchestra.run.smithi150.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-09-30T00:53:03.985 INFO:teuthology.orchestra.run.smithi150.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-09-30T00:53:04.017 INFO:teuthology.orchestra.run.smithi150.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-09-30T00:53:04.022 INFO:teuthology.orchestra.run.smithi150.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-09-30T00:53:04.025 INFO:teuthology.orchestra.run.smithi150.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-09-30T00:53:04.028 INFO:teuthology.orchestra.run.smithi150.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-09-30T00:53:04.029 INFO:teuthology.orchestra.run.smithi150.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-09-30T00:53:04.031 INFO:teuthology.orchestra.run.smithi150.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-09-30T00:53:04.036 INFO:teuthology.orchestra.run.smithi150.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-09-30T00:53:04.085 INFO:teuthology.orchestra.run.smithi150.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-09-30T00:53:04.090 INFO:teuthology.orchestra.run.smithi150.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-09-30T00:53:04.190 INFO:teuthology.orchestra.run.smithi069.stdout:Get:44 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1165-g93dce5e1-1focal [3,552 kB] 2023-09-30T00:53:04.230 INFO:teuthology.orchestra.run.smithi150.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-09-30T00:53:04.231 INFO:teuthology.orchestra.run.smithi150.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-09-30T00:53:04.232 INFO:teuthology.orchestra.run.smithi150.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-09-30T00:53:04.232 INFO:teuthology.orchestra.run.smithi150.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-09-30T00:53:04.232 INFO:teuthology.orchestra.run.smithi150.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-09-30T00:53:04.234 INFO:teuthology.orchestra.run.smithi150.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-09-30T00:53:04.302 INFO:teuthology.orchestra.run.smithi150.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-09-30T00:53:04.302 INFO:teuthology.orchestra.run.smithi150.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-09-30T00:53:04.303 INFO:teuthology.orchestra.run.smithi150.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-09-30T00:53:04.304 INFO:teuthology.orchestra.run.smithi150.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-09-30T00:53:04.305 INFO:teuthology.orchestra.run.smithi150.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-09-30T00:53:04.307 INFO:teuthology.orchestra.run.smithi150.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-09-30T00:53:04.308 INFO:teuthology.orchestra.run.smithi069.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-09-30T00:53:04.309 INFO:teuthology.orchestra.run.smithi150.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-09-30T00:53:04.310 INFO:teuthology.orchestra.run.smithi150.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-09-30T00:53:04.320 INFO:teuthology.orchestra.run.smithi150.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-09-30T00:53:04.327 INFO:teuthology.orchestra.run.smithi150.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-09-30T00:53:04.343 INFO:teuthology.orchestra.run.smithi069.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-09-30T00:53:04.348 INFO:teuthology.orchestra.run.smithi069.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-09-30T00:53:04.387 INFO:teuthology.orchestra.run.smithi069.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-09-30T00:53:04.388 INFO:teuthology.orchestra.run.smithi069.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-09-30T00:53:04.388 INFO:teuthology.orchestra.run.smithi069.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-09-30T00:53:04.389 INFO:teuthology.orchestra.run.smithi069.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-09-30T00:53:04.389 INFO:teuthology.orchestra.run.smithi069.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-09-30T00:53:04.390 INFO:teuthology.orchestra.run.smithi069.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-09-30T00:53:04.390 INFO:teuthology.orchestra.run.smithi069.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-09-30T00:53:04.392 INFO:teuthology.orchestra.run.smithi069.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-09-30T00:53:04.418 INFO:teuthology.orchestra.run.smithi069.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-09-30T00:53:04.419 INFO:teuthology.orchestra.run.smithi069.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-09-30T00:53:04.424 INFO:teuthology.orchestra.run.smithi069.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-09-30T00:53:04.427 INFO:teuthology.orchestra.run.smithi069.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-09-30T00:53:04.430 INFO:teuthology.orchestra.run.smithi069.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-09-30T00:53:04.431 INFO:teuthology.orchestra.run.smithi069.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-09-30T00:53:04.436 INFO:teuthology.orchestra.run.smithi069.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-09-30T00:53:04.441 INFO:teuthology.orchestra.run.smithi069.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-09-30T00:53:04.534 INFO:teuthology.orchestra.run.smithi069.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-09-30T00:53:04.539 INFO:teuthology.orchestra.run.smithi069.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-09-30T00:53:04.659 INFO:teuthology.orchestra.run.smithi069.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-09-30T00:53:04.660 INFO:teuthology.orchestra.run.smithi069.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-09-30T00:53:04.660 INFO:teuthology.orchestra.run.smithi069.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-09-30T00:53:04.661 INFO:teuthology.orchestra.run.smithi069.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-09-30T00:53:04.661 INFO:teuthology.orchestra.run.smithi069.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-09-30T00:53:04.662 INFO:teuthology.orchestra.run.smithi069.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-09-30T00:53:04.752 INFO:teuthology.orchestra.run.smithi069.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-09-30T00:53:04.753 INFO:teuthology.orchestra.run.smithi069.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-09-30T00:53:04.754 INFO:teuthology.orchestra.run.smithi069.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-09-30T00:53:04.755 INFO:teuthology.orchestra.run.smithi069.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-09-30T00:53:04.757 INFO:teuthology.orchestra.run.smithi069.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-09-30T00:53:04.758 INFO:teuthology.orchestra.run.smithi069.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-09-30T00:53:04.761 INFO:teuthology.orchestra.run.smithi069.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-09-30T00:53:04.761 INFO:teuthology.orchestra.run.smithi069.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-09-30T00:53:04.762 INFO:teuthology.orchestra.run.smithi069.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-09-30T00:53:04.778 INFO:teuthology.orchestra.run.smithi069.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-09-30T00:53:04.955 INFO:teuthology.orchestra.run.smithi069.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1165-g93dce5e1-1focal [758 kB] 2023-09-30T00:53:04.990 INFO:teuthology.orchestra.run.smithi150.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1165-g93dce5e1-1focal [6,453 kB] 2023-09-30T00:53:05.069 INFO:teuthology.orchestra.run.smithi069.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1165-g93dce5e1-1focal [414 kB] 2023-09-30T00:53:05.128 INFO:teuthology.orchestra.run.smithi069.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1165-g93dce5e1-1focal [22.5 kB] 2023-09-30T00:53:05.129 INFO:teuthology.orchestra.run.smithi069.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1165-g93dce5e1-1focal [200 kB] 2023-09-30T00:53:05.163 INFO:teuthology.orchestra.run.smithi150.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1165-g93dce5e1-1focal [221 kB] 2023-09-30T00:53:05.169 INFO:teuthology.orchestra.run.smithi150.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1165-g93dce5e1-1focal [153 kB] 2023-09-30T00:53:05.180 INFO:teuthology.orchestra.run.smithi069.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1165-g93dce5e1-1focal [47.1 kB] 2023-09-30T00:53:05.181 INFO:teuthology.orchestra.run.smithi069.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1165-g93dce5e1-1focal [385 kB] 2023-09-30T00:53:05.183 INFO:teuthology.orchestra.run.smithi150.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1165-g93dce5e1-1focal [1,589 kB] 2023-09-30T00:53:05.244 INFO:teuthology.orchestra.run.smithi150.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1165-g93dce5e1-1focal [7,845 kB] 2023-09-30T00:53:05.286 INFO:teuthology.orchestra.run.smithi069.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1165-g93dce5e1-1focal [5,569 kB] 2023-09-30T00:53:05.548 INFO:teuthology.orchestra.run.smithi150.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1165-g93dce5e1-1focal [28.7 MB] 2023-09-30T00:53:06.414 INFO:teuthology.orchestra.run.smithi069.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1165-g93dce5e1-1focal [116 kB] 2023-09-30T00:53:06.417 INFO:teuthology.orchestra.run.smithi069.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1165-g93dce5e1-1focal [480 kB] 2023-09-30T00:53:06.541 INFO:teuthology.orchestra.run.smithi069.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1165-g93dce5e1-1focal [27.7 MB] 2023-09-30T00:53:06.819 INFO:teuthology.orchestra.run.smithi150.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1165-g93dce5e1-1focal [3,928 B] 2023-09-30T00:53:06.820 INFO:teuthology.orchestra.run.smithi150.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1165-g93dce5e1-1focal [868 kB] 2023-09-30T00:53:06.833 INFO:teuthology.orchestra.run.smithi150.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1165-g93dce5e1-1focal [2,404 kB] 2023-09-30T00:53:06.850 INFO:teuthology.orchestra.run.smithi150.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1165-g93dce5e1-1focal [78.9 kB] 2023-09-30T00:53:06.851 INFO:teuthology.orchestra.run.smithi150.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1165-g93dce5e1-1focal [117 kB] 2023-09-30T00:53:06.852 INFO:teuthology.orchestra.run.smithi150.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1165-g93dce5e1-1focal [1,760 kB] 2023-09-30T00:53:06.888 INFO:teuthology.orchestra.run.smithi150.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1165-g93dce5e1-1focal [7,861 kB] 2023-09-30T00:53:07.196 INFO:teuthology.orchestra.run.smithi150.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1165-g93dce5e1-1focal [4,048 B] 2023-09-30T00:53:07.197 INFO:teuthology.orchestra.run.smithi150.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1165-g93dce5e1-1focal [56.2 MB] 2023-09-30T00:53:09.651 INFO:teuthology.orchestra.run.smithi150.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1165-g93dce5e1-1focal [229 kB] 2023-09-30T00:53:09.655 INFO:teuthology.orchestra.run.smithi150.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1165-g93dce5e1-1focal [21.9 kB] 2023-09-30T00:53:09.656 INFO:teuthology.orchestra.run.smithi150.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1165-g93dce5e1-1focal [17.0 MB] 2023-09-30T00:53:10.257 INFO:teuthology.orchestra.run.smithi150.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1165-g93dce5e1-1focal [79.5 kB] 2023-09-30T00:53:10.763 INFO:teuthology.orchestra.run.smithi069.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1165-g93dce5e1-1focal [6,453 kB] 2023-09-30T00:53:10.889 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 209 MB in 7s (29.1 MB/s) 2023-09-30T00:53:11.045 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-09-30T00:53:11.103 INFO:teuthology.orchestra.run.smithi150.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141758 files and directories currently installed.) 2023-09-30T00:53:11.107 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-09-30T00:53:11.119 INFO:teuthology.orchestra.run.smithi069.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1165-g93dce5e1-1focal [221 kB] 2023-09-30T00:53:11.137 INFO:teuthology.orchestra.run.smithi069.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1165-g93dce5e1-1focal [153 kB] 2023-09-30T00:53:11.139 INFO:teuthology.orchestra.run.smithi069.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1165-g93dce5e1-1focal [1,589 kB] 2023-09-30T00:53:11.227 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-30T00:53:11.241 INFO:teuthology.orchestra.run.smithi069.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1165-g93dce5e1-1focal [7,845 kB] 2023-09-30T00:53:11.572 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../001-librbd1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:11.714 INFO:teuthology.orchestra.run.smithi069.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1165-g93dce5e1-1focal [28.7 MB] 2023-09-30T00:53:11.732 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librbd1 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-30T00:53:12.436 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../002-librados2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:12.587 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librados2 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-30T00:53:13.241 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libcephfs2. 2023-09-30T00:53:13.259 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../003-libcephfs2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:13.315 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:13.357 INFO:teuthology.orchestra.run.smithi069.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1165-g93dce5e1-1focal [3,928 B] 2023-09-30T00:53:13.425 INFO:teuthology.orchestra.run.smithi069.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1165-g93dce5e1-1focal [868 kB] 2023-09-30T00:53:13.439 INFO:teuthology.orchestra.run.smithi069.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1165-g93dce5e1-1focal [2,404 kB] 2023-09-30T00:53:13.526 INFO:teuthology.orchestra.run.smithi069.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1165-g93dce5e1-1focal [78.9 kB] 2023-09-30T00:53:13.541 INFO:teuthology.orchestra.run.smithi069.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1165-g93dce5e1-1focal [117 kB] 2023-09-30T00:53:13.543 INFO:teuthology.orchestra.run.smithi069.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1165-g93dce5e1-1focal [1,760 kB] 2023-09-30T00:53:13.628 INFO:teuthology.orchestra.run.smithi069.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1165-g93dce5e1-1focal [7,861 kB] 2023-09-30T00:53:13.643 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rados. 2023-09-30T00:53:13.659 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../004-python3-rados_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:13.726 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:14.001 INFO:teuthology.orchestra.run.smithi069.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1165-g93dce5e1-1focal [4,048 B] 2023-09-30T00:53:14.001 INFO:teuthology.orchestra.run.smithi069.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1165-g93dce5e1-1focal [56.2 MB] 2023-09-30T00:53:14.054 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-09-30T00:53:14.070 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:14.112 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:14.382 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cephfs. 2023-09-30T00:53:14.401 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../006-python3-cephfs_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:14.439 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:14.742 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-ceph-common. 2023-09-30T00:53:14.759 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:14.792 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:15.095 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-prettytable. 2023-09-30T00:53:15.111 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2023-09-30T00:53:15.145 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-09-30T00:53:15.395 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rbd. 2023-09-30T00:53:15.414 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../009-python3-rbd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:15.456 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:15.834 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-09-30T00:53:15.850 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-09-30T00:53:15.892 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-30T00:53:16.195 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-09-30T00:53:16.211 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2023-09-30T00:53:16.262 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-09-30T00:53:16.548 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-09-30T00:53:16.564 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-09-30T00:53:16.606 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-30T00:53:16.740 INFO:teuthology.orchestra.run.smithi069.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1165-g93dce5e1-1focal [229 kB] 2023-09-30T00:53:16.744 INFO:teuthology.orchestra.run.smithi069.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1165-g93dce5e1-1focal [21.9 kB] 2023-09-30T00:53:16.745 INFO:teuthology.orchestra.run.smithi069.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1165-g93dce5e1-1focal [17.0 MB] 2023-09-30T00:53:16.976 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-09-30T00:53:16.992 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-09-30T00:53:17.056 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-30T00:53:17.397 INFO:teuthology.orchestra.run.smithi069.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1165-g93dce5e1-1focal [79.5 kB] 2023-09-30T00:53:17.410 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-09-30T00:53:17.427 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-09-30T00:53:17.496 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-30T00:53:17.984 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-09-30T00:53:18.000 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:53:18.024 INFO:teuthology.orchestra.run.smithi069.stdout:Fetched 209 MB in 14s (14.7 MB/s) 2023-09-30T00:53:18.053 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:53:18.148 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-09-30T00:53:18.205 INFO:teuthology.orchestra.run.smithi069.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141758 files and directories currently installed.) 2023-09-30T00:53:18.210 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-09-30T00:53:18.313 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-30T00:53:18.576 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-09-30T00:53:18.592 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:53:18.634 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:53:18.660 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../001-librbd1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:18.801 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking librbd1 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-30T00:53:18.996 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-09-30T00:53:19.013 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:53:19.054 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:53:19.441 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-09-30T00:53:19.457 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-09-30T00:53:19.475 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../002-librados2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:19.499 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-30T00:53:19.599 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking librados2 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-30T00:53:20.119 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librgw2. 2023-09-30T00:53:20.135 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../019-librgw2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:20.177 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:20.311 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libcephfs2. 2023-09-30T00:53:20.328 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../003-libcephfs2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:20.412 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:20.815 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-rados. 2023-09-30T00:53:20.833 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../004-python3-rados_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:20.865 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:20.963 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rgw. 2023-09-30T00:53:20.982 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../020-python3-rgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:21.021 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:21.243 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-09-30T00:53:21.258 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:21.301 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:21.349 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liboath0. 2023-09-30T00:53:21.369 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2023-09-30T00:53:21.399 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-09-30T00:53:21.578 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-cephfs. 2023-09-30T00:53:21.597 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../006-python3-cephfs_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:21.636 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:21.728 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libradosstriper1. 2023-09-30T00:53:21.746 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../022-libradosstriper1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:21.786 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:21.914 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-ceph-common. 2023-09-30T00:53:21.933 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:21.964 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:22.264 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-common. 2023-09-30T00:53:22.282 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../023-ceph-common_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:22.284 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-prettytable. 2023-09-30T00:53:22.301 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2023-09-30T00:53:22.322 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:22.342 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-09-30T00:53:22.578 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-rbd. 2023-09-30T00:53:22.594 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../009-python3-rbd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:22.653 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:23.005 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-09-30T00:53:23.021 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-09-30T00:53:23.072 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-30T00:53:23.375 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-09-30T00:53:23.393 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2023-09-30T00:53:23.425 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-09-30T00:53:23.786 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-09-30T00:53:23.802 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-09-30T00:53:23.844 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-30T00:53:24.205 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-09-30T00:53:24.221 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-09-30T00:53:24.288 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-30T00:53:24.599 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-09-30T00:53:24.615 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-09-30T00:53:24.649 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-30T00:53:25.070 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-base. 2023-09-30T00:53:25.085 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../024-ceph-base_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:25.138 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:25.142 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-09-30T00:53:25.158 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:53:25.202 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:53:25.715 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-09-30T00:53:25.733 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:53:25.774 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:53:26.157 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-09-30T00:53:26.166 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cryptography. 2023-09-30T00:53:26.173 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-30T00:53:26.183 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-09-30T00:53:26.215 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:53:26.223 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-30T00:53:26.627 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-openssl. 2023-09-30T00:53:26.643 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-09-30T00:53:26.644 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2023-09-30T00:53:26.660 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-09-30T00:53:26.686 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-09-30T00:53:26.701 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-30T00:53:26.956 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-09-30T00:53:26.972 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:27.014 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:27.047 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package librgw2. 2023-09-30T00:53:27.067 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../019-librgw2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:27.419 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-bcrypt. 2023-09-30T00:53:27.436 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-09-30T00:53:27.477 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-30T00:53:27.814 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cherrypy3. 2023-09-30T00:53:27.831 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-6_all.deb ... 2023-09-30T00:53:27.857 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-rgw. 2023-09-30T00:53:27.872 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-09-30T00:53:27.874 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../020-python3-rgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:27.915 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:28.242 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-natsort. 2023-09-30T00:53:28.259 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../030-python3-natsort_7.0.1-1_all.deb ... 2023-09-30T00:53:28.293 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package liboath0. 2023-09-30T00:53:28.300 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-09-30T00:53:28.309 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2023-09-30T00:53:28.351 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-09-30T00:53:28.612 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libradosstriper1. 2023-09-30T00:53:28.631 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../022-libradosstriper1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:28.653 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jwt. 2023-09-30T00:53:28.662 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:28.671 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../031-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-09-30T00:53:28.711 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-30T00:53:29.048 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-logutils. 2023-09-30T00:53:29.064 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-7_all.deb ... 2023-09-30T00:53:29.106 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-09-30T00:53:29.133 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-common. 2023-09-30T00:53:29.149 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../023-ceph-common_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:29.191 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:29.426 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-markupsafe. 2023-09-30T00:53:29.446 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../033-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-09-30T00:53:29.484 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-09-30T00:53:29.812 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-mako. 2023-09-30T00:53:29.831 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../034-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-09-30T00:53:29.870 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-30T00:53:30.216 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-simplegeneric. 2023-09-30T00:53:30.232 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-3_all.deb ... 2023-09-30T00:53:30.274 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-09-30T00:53:30.604 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-singledispatch. 2023-09-30T00:53:30.620 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-09-30T00:53:30.662 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-09-30T00:53:30.981 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-webob. 2023-09-30T00:53:30.999 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../037-python3-webob_1%3a1.8.5-2_all.deb ... 2023-09-30T00:53:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-09-30T00:53:31.367 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-soupsieve. 2023-09-30T00:53:31.386 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../038-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-09-30T00:53:31.426 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-30T00:53:31.754 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-bs4. 2023-09-30T00:53:31.771 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../039-python3-bs4_4.8.2-1_all.deb ... 2023-09-30T00:53:31.812 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-09-30T00:53:31.979 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-base. 2023-09-30T00:53:31.994 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../024-ceph-base_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:32.073 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:32.207 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-waitress. 2023-09-30T00:53:32.223 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../040-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-09-30T00:53:32.268 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-30T00:53:32.618 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-tempita. 2023-09-30T00:53:32.634 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-6_all.deb ... 2023-09-30T00:53:32.676 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-09-30T00:53:33.004 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-paste. 2023-09-30T00:53:33.020 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../042-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-09-30T00:53:33.062 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-09-30T00:53:33.083 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-cryptography. 2023-09-30T00:53:33.101 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-09-30T00:53:33.158 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-30T00:53:33.407 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-09-30T00:53:33.426 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-09-30T00:53:33.465 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-09-30T00:53:33.512 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-openssl. 2023-09-30T00:53:33.531 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2023-09-30T00:53:33.570 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-09-30T00:53:33.792 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pastedeploy. 2023-09-30T00:53:33.810 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../044-python3-pastedeploy_2.1.0-1_all.deb ... 2023-09-30T00:53:33.851 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-09-30T00:53:33.855 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-09-30T00:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:33.914 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:34.138 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-webtest. 2023-09-30T00:53:34.157 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../045-python3-webtest_2.0.32-2_all.deb ... 2023-09-30T00:53:34.221 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-09-30T00:53:34.295 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-bcrypt. 2023-09-30T00:53:34.312 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-09-30T00:53:34.353 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-30T00:53:34.532 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pecan. 2023-09-30T00:53:34.548 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../046-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-09-30T00:53:34.590 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-09-30T00:53:34.655 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-cherrypy3. 2023-09-30T00:53:34.673 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-6_all.deb ... 2023-09-30T00:53:34.714 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-09-30T00:53:34.962 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-werkzeug. 2023-09-30T00:53:34.981 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../047-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-09-30T00:53:35.041 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-30T00:53:35.084 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-natsort. 2023-09-30T00:53:35.101 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../030-python3-natsort_7.0.1-1_all.deb ... 2023-09-30T00:53:35.142 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-09-30T00:53:35.478 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-jwt. 2023-09-30T00:53:35.496 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../031-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-09-30T00:53:35.536 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-30T00:53:35.856 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-logutils. 2023-09-30T00:53:35.876 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-7_all.deb ... 2023-09-30T00:53:35.914 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-09-30T00:53:36.432 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-09-30T00:53:36.448 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:36.490 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:36.868 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-markupsafe. 2023-09-30T00:53:36.868 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr. 2023-09-30T00:53:36.884 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../049-ceph-mgr_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:36.887 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../033-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-09-30T00:53:36.926 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-09-30T00:53:36.926 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:37.253 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-mako. 2023-09-30T00:53:37.269 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../034-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-09-30T00:53:37.311 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-30T00:53:37.438 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mon. 2023-09-30T00:53:37.455 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../050-ceph-mon_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:37.496 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:37.665 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-simplegeneric. 2023-09-30T00:53:37.682 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-3_all.deb ... 2023-09-30T00:53:37.723 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-09-30T00:53:38.026 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-singledispatch. 2023-09-30T00:53:38.044 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-09-30T00:53:38.109 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-09-30T00:53:38.412 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-webob. 2023-09-30T00:53:38.431 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../037-python3-webob_1%3a1.8.5-2_all.deb ... 2023-09-30T00:53:38.470 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-09-30T00:53:38.666 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-osd. 2023-09-30T00:53:38.683 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../051-ceph-osd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:38.724 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:38.823 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-soupsieve. 2023-09-30T00:53:38.841 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../038-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-09-30T00:53:38.881 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-30T00:53:39.201 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-bs4. 2023-09-30T00:53:39.218 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../039-python3-bs4_4.8.2-1_all.deb ... 2023-09-30T00:53:39.259 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-09-30T00:53:39.612 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-waitress. 2023-09-30T00:53:39.628 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../040-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-09-30T00:53:39.673 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-30T00:53:40.032 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-tempita. 2023-09-30T00:53:40.048 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-6_all.deb ... 2023-09-30T00:53:40.090 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-09-30T00:53:40.418 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-paste. 2023-09-30T00:53:40.434 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../042-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-09-30T00:53:40.476 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-09-30T00:53:40.812 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-09-30T00:53:40.829 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-09-30T00:53:40.870 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-09-30T00:53:41.190 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-pastedeploy. 2023-09-30T00:53:41.206 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../044-python3-pastedeploy_2.1.0-1_all.deb ... 2023-09-30T00:53:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-09-30T00:53:41.469 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph. 2023-09-30T00:53:41.485 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../052-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:41.527 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:41.576 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-webtest. 2023-09-30T00:53:41.593 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../045-python3-webtest_2.0.32-2_all.deb ... 2023-09-30T00:53:41.634 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-09-30T00:53:41.880 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-fuse. 2023-09-30T00:53:41.896 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../053-ceph-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:41.938 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:41.979 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-pecan. 2023-09-30T00:53:41.999 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../046-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-09-30T00:53:42.037 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-09-30T00:53:42.375 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mds. 2023-09-30T00:53:42.375 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-werkzeug. 2023-09-30T00:53:42.391 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../047-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-09-30T00:53:42.391 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../054-ceph-mds_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:42.433 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:42.458 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-30T00:53:42.811 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-09-30T00:53:42.827 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:42.869 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:42.936 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package cephadm. 2023-09-30T00:53:42.952 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../055-cephadm_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:42.986 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:43.239 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-mgr. 2023-09-30T00:53:43.259 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../049-ceph-mgr_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:43.297 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:43.330 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jinja2. 2023-09-30T00:53:43.347 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2_all.deb ... 2023-09-30T00:53:43.388 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-09-30T00:53:43.749 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-09-30T00:53:43.765 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:43.800 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-mon. 2023-09-30T00:53:43.807 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:43.820 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../050-ceph-mon_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:43.858 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:44.178 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-repoze.lru. 2023-09-30T00:53:44.197 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-2_all.deb ... 2023-09-30T00:53:44.236 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-09-30T00:53:44.521 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libjs-underscore. 2023-09-30T00:53:44.539 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../059-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-09-30T00:53:44.580 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-30T00:53:44.890 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-09-30T00:53:44.906 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../060-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-09-30T00:53:44.948 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-30T00:53:45.012 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-osd. 2023-09-30T00:53:45.029 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../051-ceph-osd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:45.070 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:45.293 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-routes. 2023-09-30T00:53:45.311 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../061-python3-routes_2.4.1-2_all.deb ... 2023-09-30T00:53:45.351 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-09-30T00:53:45.704 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-09-30T00:53:45.720 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../062-ceph-mgr-dashboard_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:45.762 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:46.274 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-decorator. 2023-09-30T00:53:46.291 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../063-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-09-30T00:53:46.332 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-09-30T00:53:46.668 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-09-30T00:53:46.684 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../064-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-09-30T00:53:46.726 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-30T00:53:47.071 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-scipy. 2023-09-30T00:53:47.088 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../065-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-09-30T00:53:47.129 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-09-30T00:53:47.956 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph. 2023-09-30T00:53:47.973 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../052-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:48.014 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:48.275 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-fuse. 2023-09-30T00:53:48.294 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../053-ceph-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:48.325 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:48.463 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-09-30T00:53:48.480 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../066-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-09-30T00:53:48.554 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-30T00:53:48.712 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-mds. 2023-09-30T00:53:48.727 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../054-ceph-mds_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:48.761 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:49.082 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-joblib. 2023-09-30T00:53:49.102 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../067-python3-joblib_0.14.0-3_all.deb ... 2023-09-30T00:53:49.140 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-09-30T00:53:49.289 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package cephadm. 2023-09-30T00:53:49.306 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../055-cephadm_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:49.347 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:49.511 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-sklearn. 2023-09-30T00:53:49.530 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../068-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-09-30T00:53:49.577 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-30T00:53:49.692 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-jinja2. 2023-09-30T00:53:49.709 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2_all.deb ... 2023-09-30T00:53:49.750 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-09-30T00:53:50.094 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-09-30T00:53:50.110 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:50.152 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:50.302 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-09-30T00:53:50.322 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../069-ceph-mgr-diskprediction-local_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:50.360 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:50.497 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-repoze.lru. 2023-09-30T00:53:50.514 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-2_all.deb ... 2023-09-30T00:53:50.599 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-09-30T00:53:50.841 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libjs-underscore. 2023-09-30T00:53:50.860 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../059-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-09-30T00:53:50.899 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-30T00:53:51.202 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-09-30T00:53:51.218 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../060-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-09-30T00:53:51.260 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-30T00:53:51.505 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cachetools. 2023-09-30T00:53:51.523 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../070-python3-cachetools_4.0.0-1_all.deb ... 2023-09-30T00:53:51.554 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-09-30T00:53:51.570 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-routes. 2023-09-30T00:53:51.589 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../061-python3-routes_2.4.1-2_all.deb ... 2023-09-30T00:53:51.629 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-09-30T00:53:51.874 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyasn1. 2023-09-30T00:53:51.891 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-09-30T00:53:51.932 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-09-30T00:53:51.982 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-09-30T00:53:51.998 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../062-ceph-mgr-dashboard_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:52.040 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:52.277 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-09-30T00:53:52.295 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-09-30T00:53:52.335 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-30T00:53:52.705 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rsa. 2023-09-30T00:53:52.726 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../073-python3-rsa_4.0-3_all.deb ... 2023-09-30T00:53:52.763 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rsa (4.0-3) ... 2023-09-30T00:53:52.852 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-decorator. 2023-09-30T00:53:52.869 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../063-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-09-30T00:53:52.910 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-09-30T00:53:53.041 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-google-auth. 2023-09-30T00:53:53.061 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../074-python3-google-auth_1.5.1-2_all.deb ... 2023-09-30T00:53:53.124 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-09-30T00:53:53.154 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-09-30T00:53:53.170 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../064-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-09-30T00:53:53.204 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-30T00:53:53.436 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-blinker. 2023-09-30T00:53:53.452 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../075-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-09-30T00:53:53.527 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-30T00:53:53.549 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-scipy. 2023-09-30T00:53:53.565 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../065-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-09-30T00:53:53.608 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-09-30T00:53:53.838 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-oauthlib. 2023-09-30T00:53:53.855 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../076-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-09-30T00:53:53.922 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-30T00:53:54.242 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-09-30T00:53:54.262 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../077-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-09-30T00:53:54.300 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-30T00:53:54.636 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-websocket. 2023-09-30T00:53:54.652 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../078-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-09-30T00:53:54.694 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-09-30T00:53:54.857 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-09-30T00:53:54.874 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../066-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-09-30T00:53:54.915 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-30T00:53:55.039 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-kubernetes. 2023-09-30T00:53:55.056 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../079-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-09-30T00:53:55.097 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-09-30T00:53:55.435 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-joblib. 2023-09-30T00:53:55.455 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../067-python3-joblib_0.14.0-3_all.deb ... 2023-09-30T00:53:55.493 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-09-30T00:53:55.710 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-09-30T00:53:55.728 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../080-ceph-mgr-k8sevents_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:55.768 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:55.872 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-sklearn. 2023-09-30T00:53:55.889 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../068-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-09-30T00:53:55.930 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-30T00:53:56.198 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libonig5:amd64. 2023-09-30T00:53:56.216 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../081-libonig5_6.9.4-1_amd64.deb ... 2023-09-30T00:53:56.256 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-09-30T00:53:56.576 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libjq1:amd64. 2023-09-30T00:53:56.595 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../082-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-30T00:53:56.634 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-30T00:53:56.696 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-09-30T00:53:56.714 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../069-ceph-mgr-diskprediction-local_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:53:56.754 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:56.945 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package jq. 2023-09-30T00:53:56.965 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../083-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-30T00:53:57.003 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-09-30T00:53:57.281 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package socat. 2023-09-30T00:53:57.297 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../084-socat_1.7.3.3-2_amd64.deb ... 2023-09-30T00:53:57.364 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking socat (1.7.3.3-2) ... 2023-09-30T00:53:57.710 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package xmlstarlet. 2023-09-30T00:53:57.727 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-09-30T00:53:57.767 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-09-30T00:53:57.891 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-cachetools. 2023-09-30T00:53:57.906 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../070-python3-cachetools_4.0.0-1_all.deb ... 2023-09-30T00:53:57.949 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-09-30T00:53:58.139 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-test. 2023-09-30T00:53:58.156 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../086-ceph-test_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:53:58.197 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:53:58.276 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-pyasn1. 2023-09-30T00:53:58.293 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-09-30T00:53:58.334 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-09-30T00:53:58.662 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-09-30T00:53:58.681 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-09-30T00:53:58.721 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-30T00:53:59.040 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-rsa. 2023-09-30T00:53:59.060 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../073-python3-rsa_4.0-3_all.deb ... 2023-09-30T00:53:59.098 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-rsa (4.0-3) ... 2023-09-30T00:53:59.443 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-google-auth. 2023-09-30T00:53:59.463 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../074-python3-google-auth_1.5.1-2_all.deb ... 2023-09-30T00:53:59.501 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-09-30T00:53:59.804 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-blinker. 2023-09-30T00:53:59.823 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../075-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-09-30T00:53:59.904 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-30T00:54:00.206 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-oauthlib. 2023-09-30T00:54:00.225 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../076-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-09-30T00:54:00.273 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-30T00:54:00.626 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-09-30T00:54:00.645 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../077-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-09-30T00:54:00.684 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-30T00:54:01.012 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-websocket. 2023-09-30T00:54:01.029 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../078-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-09-30T00:54:01.070 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-09-30T00:54:01.381 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-kubernetes. 2023-09-30T00:54:01.398 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../079-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-09-30T00:54:01.442 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-09-30T00:54:02.005 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-09-30T00:54:02.022 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../080-ceph-mgr-k8sevents_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:54:02.088 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:02.485 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libonig5:amd64. 2023-09-30T00:54:02.504 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../081-libonig5_6.9.4-1_amd64.deb ... 2023-09-30T00:54:02.543 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-09-30T00:54:02.913 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libjq1:amd64. 2023-09-30T00:54:02.929 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../082-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-30T00:54:02.971 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-30T00:54:03.268 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-volume. 2023-09-30T00:54:03.273 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package jq. 2023-09-30T00:54:03.284 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../087-ceph-volume_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:54:03.292 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../083-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-30T00:54:03.326 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:03.331 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-09-30T00:54:03.601 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package socat. 2023-09-30T00:54:03.620 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../084-socat_1.7.3.3-2_amd64.deb ... 2023-09-30T00:54:03.664 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package fonts-mathjax. 2023-09-30T00:54:03.680 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../088-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-30T00:54:03.684 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking socat (1.7.3.3-2) ... 2023-09-30T00:54:03.722 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-30T00:54:03.988 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package xmlstarlet. 2023-09-30T00:54:04.004 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-09-30T00:54:04.046 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-09-30T00:54:04.239 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libcephfs-dev. 2023-09-30T00:54:04.258 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../089-libcephfs-dev_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:54:04.322 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:04.426 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-test. 2023-09-30T00:54:04.444 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../086-ceph-test_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:54:04.484 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:04.759 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package nvme-cli. 2023-09-30T00:54:04.776 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-09-30T00:54:04.835 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-09-30T00:54:05.181 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libjs-mathjax. 2023-09-30T00:54:05.199 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../091-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-30T00:54:05.239 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-30T00:54:06.947 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python-natsort-doc. 2023-09-30T00:54:06.966 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../092-python-natsort-doc_7.0.1-1_all.deb ... 2023-09-30T00:54:07.288 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-09-30T00:54:07.616 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-atomicwrites. 2023-09-30T00:54:07.633 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../093-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-09-30T00:54:07.716 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-09-30T00:54:08.002 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-attr. 2023-09-30T00:54:08.022 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../094-python3-attr_19.3.0-2_all.deb ... 2023-09-30T00:54:08.060 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-09-30T00:54:08.421 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-webencodings. 2023-09-30T00:54:08.438 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../095-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-09-30T00:54:08.479 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-30T00:54:08.782 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-html5lib. 2023-09-30T00:54:08.802 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../096-python3-html5lib_1.0.1-2_all.deb ... 2023-09-30T00:54:08.840 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-09-30T00:54:09.202 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-09-30T00:54:09.221 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../097-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-09-30T00:54:09.260 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-30T00:54:09.504 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package ceph-volume. 2023-09-30T00:54:09.521 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../087-ceph-volume_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-30T00:54:09.555 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:09.730 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-packaging. 2023-09-30T00:54:09.748 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../098-python3-packaging_20.3-1_all.deb ... 2023-09-30T00:54:09.796 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-packaging (20.3-1) ... 2023-09-30T00:54:09.901 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package fonts-mathjax. 2023-09-30T00:54:09.918 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../088-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-30T00:54:09.959 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-30T00:54:10.057 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pastescript. 2023-09-30T00:54:10.077 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2023-09-30T00:54:10.107 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-09-30T00:54:10.452 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pluggy. 2023-09-30T00:54:10.471 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2023-09-30T00:54:10.488 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libcephfs-dev. 2023-09-30T00:54:10.505 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../089-libcephfs-dev_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:54:10.510 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-09-30T00:54:10.546 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:10.863 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-psutil. 2023-09-30T00:54:10.880 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-09-30T00:54:10.921 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-09-30T00:54:10.932 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package nvme-cli. 2023-09-30T00:54:10.950 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-09-30T00:54:11.008 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-09-30T00:54:11.257 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-py. 2023-09-30T00:54:11.278 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-09-30T00:54:11.315 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-09-30T00:54:11.351 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package libjs-mathjax. 2023-09-30T00:54:11.372 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../091-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-30T00:54:11.434 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-30T00:54:11.660 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyinotify. 2023-09-30T00:54:11.677 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-09-30T00:54:11.718 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-30T00:54:12.071 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-wcwidth. 2023-09-30T00:54:12.088 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-09-30T00:54:12.129 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-30T00:54:12.457 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pytest. 2023-09-30T00:54:12.476 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2023-09-30T00:54:12.515 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-09-30T00:54:12.894 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-simplejson. 2023-09-30T00:54:12.912 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-09-30T00:54:12.952 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-30T00:54:13.015 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python-natsort-doc. 2023-09-30T00:54:13.034 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../092-python-natsort-doc_7.0.1-1_all.deb ... 2023-09-30T00:54:13.098 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-09-30T00:54:13.264 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-09-30T00:54:13.283 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-09-30T00:54:13.322 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-30T00:54:13.437 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-atomicwrites. 2023-09-30T00:54:13.451 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../093-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-09-30T00:54:13.492 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-09-30T00:54:13.822 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-attr. 2023-09-30T00:54:13.840 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../094-python3-attr_19.3.0-2_all.deb ... 2023-09-30T00:54:13.880 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-09-30T00:54:14.059 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package radosgw. 2023-09-30T00:54:14.078 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../108-radosgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:54:14.117 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:14.241 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-webencodings. 2023-09-30T00:54:14.260 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../095-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-09-30T00:54:14.299 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-30T00:54:14.602 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-html5lib. 2023-09-30T00:54:14.618 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../096-python3-html5lib_1.0.1-2_all.deb ... 2023-09-30T00:54:14.660 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-09-30T00:54:15.013 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-09-30T00:54:15.030 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../097-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-09-30T00:54:15.071 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-30T00:54:15.549 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-packaging. 2023-09-30T00:54:15.566 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../098-python3-packaging_20.3-1_all.deb ... 2023-09-30T00:54:15.607 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-packaging (20.3-1) ... 2023-09-30T00:54:15.846 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package rbd-fuse. 2023-09-30T00:54:15.866 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../109-rbd-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:54:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:15.927 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-pastescript. 2023-09-30T00:54:15.945 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2023-09-30T00:54:15.985 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-09-30T00:54:16.224 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-attr (19.3.0-2) ... 2023-09-30T00:54:16.339 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-pluggy. 2023-09-30T00:54:16.355 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2023-09-30T00:54:16.397 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-09-30T00:54:16.543 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-09-30T00:54:16.669 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-30T00:54:16.708 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-psutil. 2023-09-30T00:54:16.725 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-09-30T00:54:16.758 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-09-30T00:54:16.795 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-09-30T00:54:17.013 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:17.119 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-py. 2023-09-30T00:54:17.137 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-09-30T00:54:17.177 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-09-30T00:54:17.522 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-pyinotify. 2023-09-30T00:54:17.538 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-09-30T00:54:17.580 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-30T00:54:17.636 INFO:teuthology.orchestra.run.smithi150.stdout:Adding system user cephadm....done 2023-09-30T00:54:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-30T00:54:17.891 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-wcwidth. 2023-09-30T00:54:17.907 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-09-30T00:54:17.957 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-30T00:54:17.982 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-30T00:54:18.285 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-pytest. 2023-09-30T00:54:18.302 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2023-09-30T00:54:18.325 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-09-30T00:54:18.343 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-09-30T00:54:18.626 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-30T00:54:18.714 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package python3-simplejson. 2023-09-30T00:54:18.728 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-09-30T00:54:18.730 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-09-30T00:54:18.772 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-30T00:54:18.853 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-09-30T00:54:19.083 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-09-30T00:54:19.104 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-09-30T00:54:19.141 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-30T00:54:19.171 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-09-30T00:54:19.593 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-30T00:54:19.878 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package radosgw. 2023-09-30T00:54:19.892 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-09-30T00:54:19.895 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../108-radosgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:54:19.937 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:20.177 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-30T00:54:20.469 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-30T00:54:20.762 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:21.058 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-30T00:54:21.193 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-09-30T00:54:21.453 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-30T00:54:21.665 INFO:teuthology.orchestra.run.smithi069.stdout:Selecting previously unselected package rbd-fuse. 2023-09-30T00:54:21.684 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing to unpack .../109-rbd-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-30T00:54:21.723 INFO:teuthology.orchestra.run.smithi069.stdout:Unpacking rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:21.771 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-30T00:54:21.918 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-09-30T00:54:22.061 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-attr (19.3.0-2) ... 2023-09-30T00:54:22.282 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-09-30T00:54:22.353 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-09-30T00:54:22.479 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-30T00:54:22.590 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-09-30T00:54:22.606 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-09-30T00:54:22.799 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:22.866 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-30T00:54:23.237 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-09-30T00:54:23.505 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-09-30T00:54:23.866 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-09-30T00:54:23.886 INFO:teuthology.orchestra.run.smithi069.stdout:Adding system user cephadm....done 2023-09-30T00:54:24.073 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-30T00:54:24.173 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-packaging (20.3-1) ... 2023-09-30T00:54:24.202 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-30T00:54:24.449 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-09-30T00:54:24.554 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-09-30T00:54:24.575 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-30T00:54:24.676 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-30T00:54:24.838 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-30T00:54:24.936 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-09-30T00:54:24.964 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-09-30T00:54:25.116 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-09-30T00:54:25.329 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-30T00:54:25.433 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-09-30T00:54:25.697 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-30T00:54:25.835 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-30T00:54:26.099 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up socat (1.7.3.3-2) ... 2023-09-30T00:54:26.153 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-09-30T00:54:26.225 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:26.438 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-30T00:54:26.543 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:54:26.653 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-09-30T00:54:26.722 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-30T00:54:26.996 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-30T00:54:27.023 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:27.122 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-09-30T00:54:27.248 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-09-30T00:54:27.325 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-30T00:54:27.450 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-09-30T00:54:27.557 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-30T00:54:27.768 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-30T00:54:27.875 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:54:27.993 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-30T00:54:28.061 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-30T00:54:28.095 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-09-30T00:54:28.195 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-09-30T00:54:28.446 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-09-30T00:54:28.609 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-09-30T00:54:28.724 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-09-30T00:54:28.893 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-09-30T00:54:29.035 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-30T00:54:29.144 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-09-30T00:54:29.161 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-30T00:54:29.437 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-09-30T00:54:29.498 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-09-30T00:54:29.721 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-30T00:54:29.774 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-09-30T00:54:30.039 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-09-30T00:54:30.108 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-09-30T00:54:30.394 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-packaging (20.3-1) ... 2023-09-30T00:54:30.434 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-09-30T00:54:30.710 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-09-30T00:54:30.720 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-09-30T00:54:30.839 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-30T00:54:30.848 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-30T00:54:30.964 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-30T00:54:31.164 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-09-30T00:54:31.266 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-09-30T00:54:31.531 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-30T00:54:31.641 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-30T00:54:31.659 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-30T00:54:31.767 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-09-30T00:54:32.018 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-30T00:54:32.093 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-30T00:54:32.388 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librados2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:32.429 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up socat (1.7.3.3-2) ... 2023-09-30T00:54:32.514 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-09-30T00:54:32.554 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:32.906 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:54:33.040 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-09-30T00:54:33.375 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-30T00:54:33.501 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-09-30T00:54:33.627 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-09-30T00:54:33.928 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-30T00:54:33.985 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:34.112 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-30T00:54:34.237 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:54:34.238 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-09-30T00:54:34.388 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-30T00:54:34.531 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-09-30T00:54:34.623 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-09-30T00:54:34.858 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-09-30T00:54:34.940 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-09-30T00:54:35.143 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-09-30T00:54:35.242 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:54:35.368 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-09-30T00:54:35.426 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-30T00:54:35.552 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-09-30T00:54:35.727 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:35.837 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-09-30T00:54:35.853 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-30T00:54:36.154 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-30T00:54:36.188 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:36.314 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-30T00:54:36.464 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-09-30T00:54:36.640 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-30T00:54:36.871 INFO:teuthology.orchestra.run.smithi069.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-09-30T00:54:37.144 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-09-30T00:54:37.271 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-30T00:54:37.588 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-09-30T00:54:37.940 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-30T00:54:37.989 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librbd1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:38.099 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-30T00:54:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-30T00:54:38.225 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-09-30T00:54:38.290 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-09-30T00:54:38.415 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-09-30T00:54:38.541 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rsa (4.0-3) ... 2023-09-30T00:54:38.551 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-30T00:54:38.826 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-30T00:54:38.873 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up librados2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:38.960 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-09-30T00:54:38.990 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-09-30T00:54:39.253 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:39.379 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:39.624 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-30T00:54:39.625 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-30T00:54:40.088 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:40.291 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-30T00:54:40.437 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:40.564 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-30T00:54:40.610 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-09-30T00:54:40.683 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-09-30T00:54:40.903 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-09-30T00:54:41.051 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-09-30T00:54:41.263 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:41.386 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-09-30T00:54:41.390 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:41.517 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:41.667 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:41.687 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-30T00:54:41.793 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-routes (2.4.1-2) ... 2023-09-30T00:54:41.813 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-09-30T00:54:42.060 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-09-30T00:54:42.182 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:42.290 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-30T00:54:42.346 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:42.496 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:42.622 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-09-30T00:54:42.625 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:42.751 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-30T00:54:43.077 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-30T00:54:43.567 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:44.124 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up librbd1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:44.164 INFO:teuthology.orchestra.run.smithi150.stdout:Adding group ceph....done 2023-09-30T00:54:44.250 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-30T00:54:44.376 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-09-30T00:54:44.502 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-09-30T00:54:44.532 INFO:teuthology.orchestra.run.smithi150.stdout:Adding system user ceph....done 2023-09-30T00:54:44.653 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-rsa (4.0-3) ... 2023-09-30T00:54:44.708 INFO:teuthology.orchestra.run.smithi150.stdout:Setting system user ceph properties....done 2023-09-30T00:54:44.730 INFO:teuthology.orchestra.run.smithi150.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-09-30T00:54:44.876 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-09-30T00:54:44.962 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-30T00:54:45.088 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-09-30T00:54:45.182 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-09-30T00:54:45.406 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:45.532 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:45.623 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:45.752 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-30T00:54:45.753 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-30T00:54:45.772 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:46.050 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-30T00:54:46.050 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-30T00:54:46.175 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:46.395 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-30T00:54:46.674 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:46.680 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-09-30T00:54:47.007 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-09-30T00:54:47.050 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-09-30T00:54:47.367 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:47.485 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:47.509 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:47.611 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:47.737 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:47.738 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-30T00:54:47.738 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-30T00:54:47.896 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-routes (2.4.1-2) ... 2023-09-30T00:54:48.194 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:48.198 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-09-30T00:54:48.444 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-30T00:54:48.444 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-30T00:54:48.491 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:48.617 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:48.734 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-09-30T00:54:48.879 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:49.364 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-30T00:54:49.365 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-30T00:54:49.629 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:49.808 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:49.959 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:50.094 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:50.224 INFO:teuthology.orchestra.run.smithi069.stdout:Adding group ceph....done 2023-09-30T00:54:50.311 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-30T00:54:50.311 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-30T00:54:50.601 INFO:teuthology.orchestra.run.smithi069.stdout:Adding system user ceph....done 2023-09-30T00:54:50.754 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:50.784 INFO:teuthology.orchestra.run.smithi069.stdout:Setting system user ceph properties....done 2023-09-30T00:54:50.807 INFO:teuthology.orchestra.run.smithi069.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-09-30T00:54:50.896 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:50.957 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-09-30T00:54:51.022 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:51.174 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:51.273 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-09-30T00:54:51.300 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-09-30T00:54:51.452 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-09-30T00:54:51.736 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-09-30T00:54:51.739 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:51.865 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:52.134 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-30T00:54:52.135 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-30T00:54:52.809 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:53.187 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-09-30T00:54:53.644 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:53.860 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-30T00:54:53.861 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-30T00:54:54.512 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:54.557 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-09-30T00:54:54.741 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-30T00:54:54.742 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-30T00:54:55.157 INFO:teuthology.orchestra.run.smithi150.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-30T00:54:55.159 DEBUG:teuthology.parallel:result is None 2023-09-30T00:54:55.198 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:55.691 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-30T00:54:55.691 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-30T00:54:56.148 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:56.274 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:56.418 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:56.642 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-30T00:54:56.642 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-30T00:54:57.094 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:57.265 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:57.396 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:57.514 INFO:teuthology.orchestra.run.smithi069.stdout:Setting up ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-30T00:54:57.641 INFO:teuthology.orchestra.run.smithi069.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-09-30T00:54:57.819 INFO:teuthology.orchestra.run.smithi069.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-09-30T00:54:58.103 INFO:teuthology.orchestra.run.smithi069.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-09-30T00:55:00.925 INFO:teuthology.orchestra.run.smithi069.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-09-30T00:55:01.535 INFO:teuthology.orchestra.run.smithi069.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-30T00:55:01.537 DEBUG:teuthology.parallel:result is None 2023-09-30T00:55:01.539 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-30T00:55:01.688 DEBUG:teuthology.orchestra.run.smithi069:> dpkg-query -W -f '${Version}' ceph 2023-09-30T00:55:01.711 INFO:teuthology.orchestra.run.smithi069.stdout:17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:55:01.712 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:55:01.712 INFO:teuthology.task.install:The correct ceph version 17.2.6-1165-g93dce5e1-1focal is installed. 2023-09-30T00:55:01.712 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-30T00:55:01.859 DEBUG:teuthology.orchestra.run.smithi150:> dpkg-query -W -f '${Version}' ceph 2023-09-30T00:55:01.883 INFO:teuthology.orchestra.run.smithi150.stdout:17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:55:01.883 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1165-g93dce5e1-1focal 2023-09-30T00:55:01.883 INFO:teuthology.task.install:The correct ceph version 17.2.6-1165-g93dce5e1-1focal is installed. 2023-09-30T00:55:01.884 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-30T00:55:01.884 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:01.884 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T00:55:01.900 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:01.901 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T00:55:01.940 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-30T00:55:01.940 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:01.941 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T00:55:01.957 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T00:55:02.013 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:02.013 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T00:55:02.032 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T00:55:02.091 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-30T00:55:02.092 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:02.092 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T00:55:02.108 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T00:55:02.117 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:02.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T00:55:02.133 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T00:55:02.191 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-30T00:55:02.191 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:02.192 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T00:55:02.207 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T00:55:02.265 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:02.265 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T00:55:02.289 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T00:55:02.347 INFO:teuthology.run_tasks:Running task ceph... 2023-09-30T00:55:02.458 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-09-30T00:55:02.458 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /var/log/ceph 2023-09-30T00:55:02.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /var/log/ceph 2023-09-30T00:55:02.477 INFO:tasks.ceph:Disabling ceph logrotate... 2023-09-30T00:55:02.477 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-30T00:55:02.486 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-30T00:55:02.536 INFO:tasks.ceph:Creating extra log directories... 2023-09-30T00:55:02.537 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-30T00:55:02.550 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-30T00:55:02.583 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-09-30T00:55:02.584 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-09-30T00:55:02.584 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794', 'branch': 'wip-yuri4-testing-2023-09-29-0928-quincy', 'description': 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-snaps}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7406794', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri4-testing-2023-09-29-0928-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri4-testing-2023-09-29-0928-quincy', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 9810, 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri4-testing-2023-09-29-0928-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481', 'targets': {'smithi069.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481', 'cluster': 'ceph'}}, {'thrashosds': {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache target_max_objects 250', 'sudo ceph osd pool set cache min_read_recency_for_promote 2']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-09-29_19:38:56', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-09-30T00:55:02.585 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-30T00:55:02.609 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-30T00:55:02.616 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m0777 -- /var/run/ceph 2023-09-30T00:55:02.619 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/run/ceph 2023-09-30T00:55:02.635 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:02.635 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2023-09-30T00:55:02.681 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-30T00:55:02.682 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2023-09-30T00:55:02.690 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-30T00:55:02.691 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T00:55:02.691 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 359 Links: 1 2023-09-30T00:55:02.691 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T00:55:02.691 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-09-30 00:51:19.829950233 +0000 2023-09-30T00:55:02.692 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-09-30 00:51:19.553956598 +0000 2023-09-30T00:55:02.692 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-09-30 00:51:19.553956598 +0000 2023-09-30T00:55:02.692 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2023-09-30T00:55:02.693 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-30T00:55:02.750 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-09-30T00:55:02.751 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-09-30T00:55:02.751 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000410878 s, 1.2 MB/s 2023-09-30T00:55:02.753 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-30T00:55:02.762 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2023-09-30T00:55:02.770 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-30T00:55:02.770 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T00:55:02.770 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 366 Links: 1 2023-09-30T00:55:02.771 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T00:55:02.771 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-09-30 00:51:20.337938518 +0000 2023-09-30T00:55:02.771 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-09-30 00:51:20.049945159 +0000 2023-09-30T00:55:02.771 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-09-30 00:51:20.049945159 +0000 2023-09-30T00:55:02.771 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2023-09-30T00:55:02.772 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-30T00:55:02.835 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-09-30T00:55:02.835 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-09-30T00:55:02.836 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000409194 s, 1.3 MB/s 2023-09-30T00:55:02.836 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-30T00:55:02.849 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2023-09-30T00:55:02.902 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-30T00:55:02.902 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T00:55:02.902 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 372 Links: 1 2023-09-30T00:55:02.902 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T00:55:02.902 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-09-30 00:51:20.829927172 +0000 2023-09-30T00:55:02.903 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-09-30 00:51:20.557933445 +0000 2023-09-30T00:55:02.903 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-09-30 00:51:20.557933445 +0000 2023-09-30T00:55:02.903 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2023-09-30T00:55:02.903 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-30T00:55:02.916 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-09-30T00:55:02.917 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-09-30T00:55:02.917 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000385748 s, 1.3 MB/s 2023-09-30T00:55:02.919 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-30T00:55:02.971 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2023-09-30T00:55:02.978 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-30T00:55:02.978 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T00:55:02.978 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 377 Links: 1 2023-09-30T00:55:02.978 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T00:55:02.978 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-09-30 00:51:21.317915918 +0000 2023-09-30T00:55:02.978 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-09-30 00:51:21.057921914 +0000 2023-09-30T00:55:02.978 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-09-30 00:51:21.057921914 +0000 2023-09-30T00:55:02.979 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2023-09-30T00:55:02.979 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-30T00:55:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-09-30T00:55:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-09-30T00:55:02.992 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000410961 s, 1.2 MB/s 2023-09-30T00:55:02.993 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-30T00:55:03.044 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T00:55:03.045 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:03.045 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2023-09-30T00:55:03.052 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-30T00:55:03.052 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2023-09-30T00:55:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-30T00:55:03.103 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T00:55:03.103 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 358 Links: 1 2023-09-30T00:55:03.103 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T00:55:03.104 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-09-30 00:51:20.743569241 +0000 2023-09-30T00:55:03.104 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-09-30 00:51:20.739569333 +0000 2023-09-30T00:55:03.104 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-09-30 00:51:20.739569333 +0000 2023-09-30T00:55:03.104 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-09-30T00:55:03.104 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-30T00:55:03.116 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-09-30T00:55:03.116 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-09-30T00:55:03.116 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000388495 s, 1.3 MB/s 2023-09-30T00:55:03.117 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-30T00:55:03.127 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2023-09-30T00:55:03.179 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-30T00:55:03.179 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T00:55:03.179 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 367 Links: 1 2023-09-30T00:55:03.179 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T00:55:03.179 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-09-30 00:51:21.543550830 +0000 2023-09-30T00:55:03.179 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-09-30 00:51:21.247557642 +0000 2023-09-30T00:55:03.180 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-09-30 00:51:21.247557642 +0000 2023-09-30T00:55:03.180 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-09-30T00:55:03.180 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-30T00:55:03.192 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-09-30T00:55:03.192 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-09-30T00:55:03.192 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000395998 s, 1.3 MB/s 2023-09-30T00:55:03.193 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-30T00:55:03.201 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2023-09-30T00:55:03.210 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-30T00:55:03.210 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T00:55:03.210 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 374 Links: 1 2023-09-30T00:55:03.210 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T00:55:03.210 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-09-30 00:51:21.795545032 +0000 2023-09-30T00:55:03.211 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-09-30 00:51:21.791545123 +0000 2023-09-30T00:55:03.211 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-09-30 00:51:21.791545123 +0000 2023-09-30T00:55:03.211 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-09-30T00:55:03.211 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-30T00:55:03.266 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-09-30T00:55:03.267 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-09-30T00:55:03.267 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000393095 s, 1.3 MB/s 2023-09-30T00:55:03.268 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-30T00:55:03.277 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2023-09-30T00:55:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-30T00:55:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T00:55:03.336 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 378 Links: 1 2023-09-30T00:55:03.336 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T00:55:03.336 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-09-30 00:51:22.551527634 +0000 2023-09-30T00:55:03.336 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-09-30 00:51:22.295533525 +0000 2023-09-30T00:55:03.336 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-09-30 00:51:22.295533525 +0000 2023-09-30T00:55:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-09-30T00:55:03.336 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-30T00:55:03.347 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-09-30T00:55:03.348 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-09-30T00:55:03.348 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000406788 s, 1.3 MB/s 2023-09-30T00:55:03.349 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-30T00:55:03.401 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-30T00:55:03.402 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi069.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-30T00:55:03.402 INFO:tasks.ceph:Generating config... 2023-09-30T00:55:03.404 INFO:tasks.ceph:[osd] debug monc = 20 2023-09-30T00:55:03.404 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-09-30T00:55:03.405 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2023-09-30T00:55:03.405 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-09-30T00:55:03.406 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-09-30T00:55:03.406 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-09-30T00:55:03.406 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-09-30T00:55:03.406 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-09-30T00:55:03.406 INFO:tasks.ceph:[osd] debug ms = 1 2023-09-30T00:55:03.406 INFO:tasks.ceph:[osd] debug osd = 20 2023-09-30T00:55:03.406 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-09-30T00:55:03.407 INFO:tasks.ceph:[osd] filestore odsync write = True 2023-09-30T00:55:03.407 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-09-30T00:55:03.407 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-09-30T00:55:03.407 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-09-30T00:55:03.407 INFO:tasks.ceph:[osd] osd backoff on degraded = True 2023-09-30T00:55:03.407 INFO:tasks.ceph:[osd] osd backoff on peering = True 2023-09-30T00:55:03.408 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-09-30T00:55:03.408 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-09-30T00:55:03.408 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-09-30T00:55:03.408 INFO:tasks.ceph:[osd] osd max backfills = 2 2023-09-30T00:55:03.408 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-09-30T00:55:03.408 INFO:tasks.ceph:[osd] osd max scrubs = 3 2023-09-30T00:55:03.409 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-09-30T00:55:03.409 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2023-09-30T00:55:03.409 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-09-30T00:55:03.409 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-09-30T00:55:03.409 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-09-30T00:55:03.409 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-09-30T00:55:03.410 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-09-30T00:55:03.410 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-09-30T00:55:03.410 INFO:tasks.ceph:[osd] osd snap trim sleep = 0.5 2023-09-30T00:55:03.411 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-09-30T00:55:03.411 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-09-30T00:55:03.411 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-09-30T00:55:03.411 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-09-30T00:55:03.412 INFO:tasks.ceph:[global] ms inject delay max = 1 2023-09-30T00:55:03.412 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2023-09-30T00:55:03.412 INFO:tasks.ceph:[global] ms inject delay type = osd 2023-09-30T00:55:03.412 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2023-09-30T00:55:03.412 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2023-09-30T00:55:03.413 INFO:tasks.ceph:[global] ms type = async 2023-09-30T00:55:03.413 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 2 2023-09-30T00:55:03.413 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 1 2023-09-30T00:55:03.415 INFO:tasks.ceph:[global] osd_pg_log_trim_min = 0 2023-09-30T00:55:03.415 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-09-30T00:55:03.415 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2023-09-30T00:55:03.416 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-09-30T00:55:03.416 INFO:tasks.ceph:[mgr] debug ms = 1 2023-09-30T00:55:03.416 INFO:tasks.ceph:[mon] debug mon = 20 2023-09-30T00:55:03.416 INFO:tasks.ceph:[mon] debug ms = 1 2023-09-30T00:55:03.416 INFO:tasks.ceph:[mon] debug paxos = 20 2023-09-30T00:55:03.416 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-09-30T00:55:03.416 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-09-30T00:55:03.416 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-09-30T00:55:03.416 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-09-30T00:55:03.416 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-09-30T00:55:03.417 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-09-30T00:55:03.417 INFO:tasks.ceph:Setting up mon.a... 2023-09-30T00:55:03.417 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-09-30T00:55:03.450 INFO:teuthology.orchestra.run.smithi069.stdout:creating /etc/ceph/ceph.keyring 2023-09-30T00:55:03.453 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-09-30T00:55:03.488 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T00:55:03.545 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.69'), ('mon.c', '[v2:172.21.15.69:3301,v1:172.21.15.69:6790]'), ('mon.b', '172.21.15.150')] 2023-09-30T00:55:03.546 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.69,[v2:172.21.15.69:3301,v1:172.21.15.69:6790],172.21.15.150', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-09-30T00:55:03.547 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:03.548 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-09-30T00:55:03.553 DEBUG:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.69 --addv c '[v2:172.21.15.69:3301,v1:172.21.15.69:6790]' --add b 172.21.15.150 --print /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:03.634 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:03.634 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool: generated fsid a4f1df1b-c238-45bb-b595-c54d4bb08739 2023-09-30T00:55:03.634 INFO:teuthology.orchestra.run.smithi069.stdout:setting min_mon_release = octopus 2023-09-30T00:55:03.634 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 0 2023-09-30T00:55:03.634 INFO:teuthology.orchestra.run.smithi069.stdout:fsid a4f1df1b-c238-45bb-b595-c54d4bb08739 2023-09-30T00:55:03.634 INFO:teuthology.orchestra.run.smithi069.stdout:last_changed 2023-09-30T00:55:03.634038+0000 2023-09-30T00:55:03.635 INFO:teuthology.orchestra.run.smithi069.stdout:created 2023-09-30T00:55:03.634038+0000 2023-09-30T00:55:03.635 INFO:teuthology.orchestra.run.smithi069.stdout:min_mon_release 15 (octopus) 2023-09-30T00:55:03.635 INFO:teuthology.orchestra.run.smithi069.stdout:election_strategy: 3 2023-09-30T00:55:03.635 INFO:teuthology.orchestra.run.smithi069.stdout:0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2023-09-30T00:55:03.635 INFO:teuthology.orchestra.run.smithi069.stdout:1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2023-09-30T00:55:03.635 INFO:teuthology.orchestra.run.smithi069.stdout:2: [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] mon.c 2023-09-30T00:55:03.635 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-09-30T00:55:03.636 DEBUG:teuthology.orchestra.run.smithi069:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-09-30T00:55:03.641 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a4f1df1b-c238-45bb-b595-c54d4bb08739... 2023-09-30T00:55:03.642 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-30T00:55:03.644 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-30T00:55:03.675 INFO:teuthology.orchestra.run.smithi069.stdout:[global] 2023-09-30T00:55:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: chdir = "" 2023-09-30T00:55:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-30T00:55:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: auth supported = cephx 2023-09-30T00:55:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: filestore xattr use omap = true 2023-09-30T00:55:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: mon clock drift allowed = 1.000 2023-09-30T00:55:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: osd crush chooseleaf type = 0 2023-09-30T00:55:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: auth debug = true 2023-09-30T00:55:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: ms die on old message = true 2023-09-30T00:55:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: ms die on bug = true 2023-09-30T00:55:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-30T00:55:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: mon pg warn max object skew = 0 2023-09-30T00:55:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: # disable pg_autoscaler by default for new pools 2023-09-30T00:55:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-30T00:55:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: osd pool default size = 2 2023-09-30T00:55:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd allow primary affinity = true 2023-09-30T00:55:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd allow pg remap = true 2023-09-30T00:55:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on legacy crush tunables = false 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on crush straw calc version zero = false 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on no sortbitwise = false 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on osd down out interval zero = false 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on too few osds = false 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: mon_allow_pool_size_one = true 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: osd default data pool replay window = 5 2023-09-30T00:55:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: mon allow pool delete = true 2023-09-30T00:55:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: mon cluster log file level = debug 2023-09-30T00:55:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: debug asserts on shutdown = true 2023-09-30T00:55:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: mon health detail to clog = false 2023-09-30T00:55:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: mon host = "172.21.15.69,[v2:172.21.15.69:3301,v1:172.21.15.69:6790],172.21.15.150" 2023-09-30T00:55:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: mon client directed command retry = 5 2023-09-30T00:55:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: mon election default strategy = 3 2023-09-30T00:55:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: ms inject delay max = 1 2023-09-30T00:55:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: ms inject delay probability = 0.005 2023-09-30T00:55:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: ms inject delay type = osd 2023-09-30T00:55:03.681 INFO:teuthology.orchestra.run.smithi069.stdout: ms inject internal delays = 0.002 2023-09-30T00:55:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: ms inject socket failures = 2500 2023-09-30T00:55:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: ms type = async 2023-09-30T00:55:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: osd_max_pg_log_entries = 2 2023-09-30T00:55:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: osd_min_pg_log_entries = 1 2023-09-30T00:55:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: osd_pg_log_trim_min = 0 2023-09-30T00:55:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: osd_pool_default_min_size = 2 2023-09-30T00:55:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: osd_pool_default_size = 3 2023-09-30T00:55:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: fsid = a4f1df1b-c238-45bb-b595-c54d4bb08739 2023-09-30T00:55:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.682 INFO:teuthology.orchestra.run.smithi069.stdout:[osd] 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: osd journal size = 100 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: osd scrub load threshold = 5.0 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: osd scrub max interval = 120 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: osd mclock profile = high_recovery_ops 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: osd recover clone overlap = true 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: osd recovery max chunk = 1048576 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug shutdown = true 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug op order = true 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug verify stray on activate = true 2023-09-30T00:55:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: osd open classes on start = true 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug pg log writeout = true 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: osd deep scrub update digest min age = 30 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: osd map max advance = 10 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: journal zero on create = true 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: filestore ondisk finisher threads = 3 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: filestore apply finisher threads = 3 2023-09-30T00:55:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: bdev debug aio = true 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug misdirected ops = true 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: debug monc = 20 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: bluestore block size = 96636764160 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: bluestore compression algorithm = snappy 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: bluestore compression mode = aggressive 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: bluestore fsck on mount = True 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: bluestore zero block detection = True 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: debug bluefs = 20 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: debug bluestore = 20 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2023-09-30T00:55:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: debug osd = 20 2023-09-30T00:55:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: debug rocksdb = 10 2023-09-30T00:55:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: filestore odsync write = True 2023-09-30T00:55:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd backfillfull_ratio = 0.85 2023-09-30T00:55:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd full ratio = 0.9 2023-09-30T00:55:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd nearfull ratio = 0.8 2023-09-30T00:55:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: osd backoff on degraded = True 2023-09-30T00:55:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: osd backoff on peering = True 2023-09-30T00:55:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug verify cached snaps = True 2023-09-30T00:55:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug verify missing on start = True 2023-09-30T00:55:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: osd failsafe full ratio = 0.95 2023-09-30T00:55:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: osd max backfills = 2 2023-09-30T00:55:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: osd max markdown count = 1000 2023-09-30T00:55:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: osd max scrubs = 3 2023-09-30T00:55:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: osd mclock override recovery settings = True 2023-09-30T00:55:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: osd objectstore = bluestore 2023-09-30T00:55:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: osd op queue = debug_random 2023-09-30T00:55:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: osd op queue cut off = debug_random 2023-09-30T00:55:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: osd scrub min interval = 60 2023-09-30T00:55:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: osd shutdown pgref assert = True 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: osd snap trim sleep = 0.5 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: osd_max_pg_log_entries = 3000 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: osd_min_pg_log_entries = 3000 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout:[mgr] 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: debug mgr = 20 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: debug mon = 20 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: debug auth = 20 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min pgs per osd = 4 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min bytes per osd = 10 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: mgr/telemetry/nag = false 2023-09-30T00:55:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.689 INFO:teuthology.orchestra.run.smithi069.stdout:[mon] 2023-09-30T00:55:03.689 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2023-09-30T00:55:03.689 INFO:teuthology.orchestra.run.smithi069.stdout: debug mon = 20 2023-09-30T00:55:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: debug paxos = 20 2023-09-30T00:55:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: debug auth = 20 2023-09-30T00:55:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: mon data avail warn = 5 2023-09-30T00:55:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: mon mgr mkfs grace = 240 2023-09-30T00:55:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min pgs per osd = 4 2023-09-30T00:55:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd reporter subtree level = osd 2023-09-30T00:55:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd prime pg temp = true 2023-09-30T00:55:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min bytes per osd = 10 2023-09-30T00:55:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-30T00:55:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: auth mon ticket ttl = 660 # 11m 2023-09-30T00:55:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: auth service ticket ttl = 240 # 4m 2023-09-30T00:55:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: # don't complain about insecure global_id in the test suite 2023-09-30T00:55:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-30T00:55:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-30T00:55:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: # 1m isn't quite enough 2023-09-30T00:55:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: mon_down_mkfs_grace = 2m 2023-09-30T00:55:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_filestore_osds = false 2023-09-30T00:55:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: mon min osdmap epochs = 50 2023-09-30T00:55:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: mon osdmap full prune interval = 2 2023-09-30T00:55:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: mon osdmap full prune min = 15 2023-09-30T00:55:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: mon osdmap full prune txsize = 2 2023-09-30T00:55:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: mon scrub interval = 300 2023-09-30T00:55:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: paxos service trim min = 10 2023-09-30T00:55:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:03.692 INFO:teuthology.orchestra.run.smithi069.stdout:[client] 2023-09-30T00:55:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: rgw cache enabled = true 2023-09-30T00:55:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: rgw enable ops log = true 2023-09-30T00:55:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: rgw enable usage log = true 2023-09-30T00:55:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-30T00:55:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-30T00:55:03.693 INFO:teuthology.orchestra.run.smithi069.stdout:[mon.a] 2023-09-30T00:55:03.693 INFO:teuthology.orchestra.run.smithi069.stdout:[mon.c] 2023-09-30T00:55:03.693 INFO:teuthology.orchestra.run.smithi069.stdout:[mon.b] 2023-09-30T00:55:03.695 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2023-09-30T00:55:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2023-09-30T00:55:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-30T00:55:03.695 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2023-09-30T00:55:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2023-09-30T00:55:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2023-09-30T00:55:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2023-09-30T00:55:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2023-09-30T00:55:03.696 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2023-09-30T00:55:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2023-09-30T00:55:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-30T00:55:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2023-09-30T00:55:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2023-09-30T00:55:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-30T00:55:03.697 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2023-09-30T00:55:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2023-09-30T00:55:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2023-09-30T00:55:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2023-09-30T00:55:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2023-09-30T00:55:03.698 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2023-09-30T00:55:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2023-09-30T00:55:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2023-09-30T00:55:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-30T00:55:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-30T00:55:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2023-09-30T00:55:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-30T00:55:03.699 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2023-09-30T00:55:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2023-09-30T00:55:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2023-09-30T00:55:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2023-09-30T00:55:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2023-09-30T00:55:03.700 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.69,[v2:172.21.15.69:3301,v1:172.21.15.69:6790],172.21.15.150" 2023-09-30T00:55:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: mon client directed command retry = 5 2023-09-30T00:55:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: mon election default strategy = 3 2023-09-30T00:55:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: ms inject delay max = 1 2023-09-30T00:55:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: ms inject delay probability = 0.005 2023-09-30T00:55:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: ms inject delay type = osd 2023-09-30T00:55:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: ms inject internal delays = 0.002 2023-09-30T00:55:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: ms inject socket failures = 2500 2023-09-30T00:55:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: ms type = async 2023-09-30T00:55:03.701 INFO:teuthology.orchestra.run.smithi150.stdout: osd_max_pg_log_entries = 2 2023-09-30T00:55:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: osd_min_pg_log_entries = 1 2023-09-30T00:55:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pg_log_trim_min = 0 2023-09-30T00:55:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_min_size = 2 2023-09-30T00:55:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_size = 3 2023-09-30T00:55:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = a4f1df1b-c238-45bb-b595-c54d4bb08739 2023-09-30T00:55:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.702 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2023-09-30T00:55:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2023-09-30T00:55:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.702 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 120 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2023-09-30T00:55:03.703 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2023-09-30T00:55:03.704 INFO:teuthology.orchestra.run.smithi150.stdout: debug monc = 20 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore compression algorithm = snappy 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore compression mode = aggressive 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore zero block detection = True 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 20 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 20 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 10 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: filestore odsync write = True 2023-09-30T00:55:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2023-09-30T00:55:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2023-09-30T00:55:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2023-09-30T00:55:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: osd backoff on degraded = True 2023-09-30T00:55:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: osd backoff on peering = True 2023-09-30T00:55:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify cached snaps = True 2023-09-30T00:55:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify missing on start = True 2023-09-30T00:55:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2023-09-30T00:55:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: osd max backfills = 2 2023-09-30T00:55:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: osd max markdown count = 1000 2023-09-30T00:55:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: osd max scrubs = 3 2023-09-30T00:55:03.706 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock override recovery settings = True 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: osd op queue = debug_random 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: osd op queue cut off = debug_random 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub min interval = 60 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: osd shutdown pgref assert = True 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: osd snap trim sleep = 0.5 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: osd_max_pg_log_entries = 3000 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: osd_min_pg_log_entries = 3000 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-09-30T00:55:03.707 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-09-30T00:55:03.708 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2023-09-30T00:55:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2023-09-30T00:55:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-09-30T00:55:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2023-09-30T00:55:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2023-09-30T00:55:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-09-30T00:55:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-30T00:55:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2023-09-30T00:55:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2023-09-30T00:55:03.709 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: mon min osdmap epochs = 50 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: mon osdmap full prune interval = 2 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: mon osdmap full prune min = 15 2023-09-30T00:55:03.710 INFO:teuthology.orchestra.run.smithi150.stdout: mon osdmap full prune txsize = 2 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: mon scrub interval = 300 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: paxos service trim min = 10 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2023-09-30T00:55:03.711 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2023-09-30T00:55:03.712 INFO:tasks.ceph:Creating admin key on mon.a... 2023-09-30T00:55:03.712 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-09-30T00:55:03.784 INFO:tasks.ceph:Copying monmap to all nodes... 2023-09-30T00:55:03.784 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:03.785 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-09-30T00:55:03.791 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:03.791 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-09-30T00:55:03.798 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi069.front.sepia.ceph.com 2023-09-30T00:55:03.798 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:03.798 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-30T00:55:03.798 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T00:55:03.822 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:03.822 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:03.872 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi150.front.sepia.ceph.com 2023-09-30T00:55:03.880 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:03.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-30T00:55:03.882 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T00:55:03.909 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:03.909 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:03.917 INFO:tasks.ceph:Setting up mon nodes... 2023-09-30T00:55:03.917 INFO:tasks.ceph:Setting up mgr nodes... 2023-09-30T00:55:03.917 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-09-30T00:55:03.964 INFO:teuthology.orchestra.run.smithi069.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-09-30T00:55:03.967 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-09-30T00:55:04.013 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-09-30T00:55:04.015 INFO:tasks.ceph:Setting up mds nodes... 2023-09-30T00:55:04.016 INFO:tasks.ceph_client:Setting up client nodes... 2023-09-30T00:55:04.016 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-30T00:55:04.051 INFO:teuthology.orchestra.run.smithi069.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-09-30T00:55:04.064 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-09-30T00:55:04.101 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-09-30T00:55:04.114 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-09-30T00:55:04.114 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi069.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-30T00:55:04.115 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-09-30T00:55:04.130 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T00:55:04.130 INFO:tasks.ceph:role: osd.0 2023-09-30T00:55:04.130 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi069.front.sepia.ceph.com 2023-09-30T00:55:04.130 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-30T00:55:04.623 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T00:55:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T00:55:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T00:55:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 2023-09-30T00:55:04.624 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T00:55:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2023-09-30T00:55:04.624 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T00:55:04.624 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T00:55:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T00:55:04.624 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T00:55:04.626 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2023-09-30T00:55:04.626 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-09-30T00:55:04.701 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-09-30T00:55:04.714 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T00:55:04.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:55:04.717 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-09-30T00:55:04.774 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T00:55:04.774 INFO:tasks.ceph:role: osd.1 2023-09-30T00:55:04.774 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi069.front.sepia.ceph.com 2023-09-30T00:55:04.774 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-30T00:55:05.293 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T00:55:05.293 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T00:55:05.293 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T00:55:05.293 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 2023-09-30T00:55:05.293 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T00:55:05.294 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2023-09-30T00:55:05.294 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T00:55:05.294 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T00:55:05.294 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T00:55:05.294 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T00:55:05.295 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2023-09-30T00:55:05.295 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-09-30T00:55:05.344 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-09-30T00:55:05.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:55:05.357 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T00:55:05.357 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-09-30T00:55:05.414 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T00:55:05.414 INFO:tasks.ceph:role: osd.2 2023-09-30T00:55:05.414 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi069.front.sepia.ceph.com 2023-09-30T00:55:05.414 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-30T00:55:05.880 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T00:55:05.881 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T00:55:05.881 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T00:55:05.881 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 2023-09-30T00:55:05.881 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T00:55:05.881 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2023-09-30T00:55:05.881 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T00:55:05.881 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T00:55:05.881 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T00:55:05.881 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T00:55:05.883 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2023-09-30T00:55:05.883 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-09-30T00:55:05.940 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-09-30T00:55:05.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:55:05.952 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T00:55:05.952 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-09-30T00:55:05.968 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T00:55:05.968 INFO:tasks.ceph:role: osd.3 2023-09-30T00:55:05.968 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi069.front.sepia.ceph.com 2023-09-30T00:55:05.968 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-30T00:55:06.483 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T00:55:06.483 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T00:55:06.484 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T00:55:06.484 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 2023-09-30T00:55:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T00:55:06.484 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2023-09-30T00:55:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T00:55:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T00:55:06.484 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T00:55:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T00:55:06.485 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2023-09-30T00:55:06.486 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-09-30T00:55:06.547 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-09-30T00:55:06.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:55:06.559 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T00:55:06.559 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:06.652 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:06.647+0000 7fe54aa133c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-09-30T00:55:06.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:06.651+0000 7fe54aa133c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-09-30T00:55:06.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:06.651+0000 7fe54aa133c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T00:55:06.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:06.651+0000 7fe54aa133c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T00:55:06.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:06.651+0000 7fe54aa133c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T00:55:06.659 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:06.655+0000 7fe54aa133c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-09-30T00:55:07.475 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:07.471+0000 7fe54aa133c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:08.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:08.255+0000 7fe54aa133c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:09.040 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:09.035+0000 7fe54aa133c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:09.907 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-09-30T00:55:09.923 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:10.003+0000 7fe2db4c43c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-09-30T00:55:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:10.003+0000 7fe2db4c43c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-09-30T00:55:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:10.003+0000 7fe2db4c43c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T00:55:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:10.003+0000 7fe2db4c43c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T00:55:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:10.003+0000 7fe2db4c43c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T00:55:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:10.003+0000 7fe2db4c43c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-09-30T00:55:10.568 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:10.563+0000 7fe2db4c43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:11.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:11.343+0000 7fe2db4c43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:12.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:12.123+0000 7fe2db4c43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:12.995 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-09-30T00:55:13.011 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:13.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:13.091+0000 7f66bae883c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-09-30T00:55:13.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:13.091+0000 7f66bae883c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-09-30T00:55:13.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:13.091+0000 7f66bae883c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T00:55:13.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:13.091+0000 7f66bae883c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T00:55:13.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:13.091+0000 7f66bae883c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T00:55:13.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:13.091+0000 7f66bae883c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-09-30T00:55:13.663 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:13.659+0000 7f66bae883c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:14.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:14.443+0000 7f66bae883c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:15.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:15.235+0000 7f66bae883c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:16.101 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-09-30T00:55:16.116 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:16.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:16.195+0000 7f7a226d93c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-09-30T00:55:16.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:16.199+0000 7f7a226d93c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-09-30T00:55:16.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:16.199+0000 7f7a226d93c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T00:55:16.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:16.199+0000 7f7a226d93c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T00:55:16.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:16.199+0000 7f7a226d93c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T00:55:16.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:16.199+0000 7f7a226d93c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-09-30T00:55:16.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:16.751+0000 7f7a226d93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:17.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:17.535+0000 7f7a226d93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:18.323 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T00:55:18.319+0000 7f7a226d93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:19.185 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-09-30T00:55:19.202 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-09-30T00:55:19.219 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-30T00:55:19.219 INFO:tasks.ceph:role: osd.4 2023-09-30T00:55:19.220 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com 2023-09-30T00:55:19.220 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-30T00:55:19.785 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T00:55:19.785 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T00:55:19.785 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T00:55:19.785 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 2023-09-30T00:55:19.785 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T00:55:19.785 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-09-30T00:55:19.785 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T00:55:19.785 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T00:55:19.786 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T00:55:19.786 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T00:55:19.787 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-09-30T00:55:19.787 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-09-30T00:55:19.869 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-09-30T00:55:19.881 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T00:55:19.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:55:19.883 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-09-30T00:55:19.938 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-30T00:55:19.938 INFO:tasks.ceph:role: osd.5 2023-09-30T00:55:19.938 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com 2023-09-30T00:55:19.938 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-30T00:55:20.451 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T00:55:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T00:55:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T00:55:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 2023-09-30T00:55:20.451 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T00:55:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-09-30T00:55:20.451 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T00:55:20.451 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T00:55:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T00:55:20.451 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T00:55:20.454 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-09-30T00:55:20.454 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-09-30T00:55:20.496 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-09-30T00:55:20.506 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T00:55:20.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:55:20.507 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-09-30T00:55:20.564 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-30T00:55:20.564 INFO:tasks.ceph:role: osd.6 2023-09-30T00:55:20.564 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com 2023-09-30T00:55:20.565 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-30T00:55:21.050 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T00:55:21.050 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T00:55:21.050 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T00:55:21.050 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 2023-09-30T00:55:21.050 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T00:55:21.050 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-09-30T00:55:21.051 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T00:55:21.051 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T00:55:21.051 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T00:55:21.051 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T00:55:21.052 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-09-30T00:55:21.052 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-09-30T00:55:21.096 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-09-30T00:55:21.152 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T00:55:21.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:55:21.153 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-09-30T00:55:21.163 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-30T00:55:21.163 INFO:tasks.ceph:role: osd.7 2023-09-30T00:55:21.163 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com 2023-09-30T00:55:21.163 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-30T00:55:21.657 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T00:55:21.657 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T00:55:21.657 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T00:55:21.657 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 2023-09-30T00:55:21.657 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T00:55:21.658 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-09-30T00:55:21.658 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T00:55:21.658 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T00:55:21.658 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T00:55:21.658 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T00:55:21.659 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-09-30T00:55:21.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-09-30T00:55:21.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-09-30T00:55:21.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T00:55:21.727 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2023-09-30T00:55:21.728 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:21.840 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:21.833+0000 7f21aa2a83c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-09-30T00:55:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:21.837+0000 7f21aa2a83c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-09-30T00:55:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:21.837+0000 7f21aa2a83c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-30T00:55:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:21.837+0000 7f21aa2a83c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-30T00:55:21.844 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:21.837+0000 7f21aa2a83c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-30T00:55:21.852 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:21.845+0000 7f21aa2a83c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-09-30T00:55:22.665 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:22.661+0000 7f21aa2a83c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:23.445 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:23.441+0000 7f21aa2a83c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:24.227 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:24.221+0000 7f21aa2a83c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:25.077 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-09-30T00:55:25.094 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:25.133 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:25.129+0000 7f14d3c903c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-09-30T00:55:25.134 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:25.129+0000 7f14d3c903c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-09-30T00:55:25.134 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:25.129+0000 7f14d3c903c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-30T00:55:25.134 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:25.129+0000 7f14d3c903c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-30T00:55:25.134 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:25.129+0000 7f14d3c903c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-30T00:55:25.134 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:25.129+0000 7f14d3c903c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-09-30T00:55:25.696 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:25.689+0000 7f14d3c903c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:26.475 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:26.469+0000 7f14d3c903c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:27.256 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:27.253+0000 7f14d3c903c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:28.124 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-09-30T00:55:28.141 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:28.227 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:28.221+0000 7f087bd543c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-09-30T00:55:28.556 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:28.221+0000 7f087bd543c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-09-30T00:55:28.556 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:28.221+0000 7f087bd543c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-30T00:55:28.556 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:28.221+0000 7f087bd543c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-30T00:55:28.556 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:28.221+0000 7f087bd543c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-30T00:55:28.557 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:28.221+0000 7f087bd543c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-09-30T00:55:28.801 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:28.797+0000 7f087bd543c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:29.583 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:29.577+0000 7f087bd543c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:30.373 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:30.369+0000 7f087bd543c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:31.220 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-09-30T00:55:31.233 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:31.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:31.313+0000 7fb8561363c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-09-30T00:55:31.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:31.313+0000 7fb8561363c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-09-30T00:55:31.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:31.313+0000 7fb8561363c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-30T00:55:31.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:31.313+0000 7fb8561363c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-30T00:55:31.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:31.313+0000 7fb8561363c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-30T00:55:31.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:31.317+0000 7fb8561363c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-09-30T00:55:31.887 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:31.881+0000 7fb8561363c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:32.676 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:32.673+0000 7fb8561363c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:33.455 INFO:teuthology.orchestra.run.smithi150.stderr:2023-09-30T00:55:33.449+0000 7fb8561363c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-30T00:55:34.324 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-09-30T00:55:34.340 INFO:tasks.ceph:Reading keys from all nodes... 2023-09-30T00:55:34.341 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:34.341 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-09-30T00:55:34.357 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:34.357 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-09-30T00:55:34.411 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:34.411 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-09-30T00:55:34.427 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:34.427 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-09-30T00:55:34.441 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:34.442 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-09-30T00:55:34.499 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:34.499 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-09-30T00:55:34.514 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:34.514 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-09-30T00:55:34.571 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:34.571 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-09-30T00:55:34.587 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:34.587 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-09-30T00:55:34.600 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:34.600 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-09-30T00:55:34.663 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:34.663 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-09-30T00:55:34.670 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:34.670 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-09-30T00:55:34.677 INFO:tasks.ceph:Adding keys to all mons... 2023-09-30T00:55:34.677 DEBUG:teuthology.orchestra.run.smithi069:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-30T00:55:34.716 DEBUG:teuthology.orchestra.run.smithi150:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-30T00:55:34.728 INFO:teuthology.orchestra.run.smithi069.stdout:[mgr.y] 2023-09-30T00:55:34.728 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDncRdl/l95ORAAti+3wYycUMxqMD27dzvhyA== 2023-09-30T00:55:34.728 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.0] 2023-09-30T00:55:34.728 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDqcRdlgpLlJhAAM8Z599+drzwwsaSHh2YT/A== 2023-09-30T00:55:34.728 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.1] 2023-09-30T00:55:34.728 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDucRdliUKAABAAZdwBx+vp7jsGTlVcuVJbeg== 2023-09-30T00:55:34.728 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.2] 2023-09-30T00:55:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDxcRdlg4+3BRAAfdk78FFad4T3XIFrD92vgw== 2023-09-30T00:55:34.729 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.3] 2023-09-30T00:55:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQD0cRdlu9cNDBAAuaQgLP5tKVm+kZtP2Kl5/A== 2023-09-30T00:55:34.729 INFO:teuthology.orchestra.run.smithi069.stdout:[mgr.x] 2023-09-30T00:55:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDocRdlaGzBABAARNlrZaKMlNvDnAhC3WorMA== 2023-09-30T00:55:34.729 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.4] 2023-09-30T00:55:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQD5cRdlOMUPMhAAPpvfe0HbhG6Nsw97HmjT9A== 2023-09-30T00:55:34.729 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.5] 2023-09-30T00:55:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQD9cRdlBOjuBxAAgYnWh7eHq26T073sPfxUwQ== 2023-09-30T00:55:34.729 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.6] 2023-09-30T00:55:34.729 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAAchdlDieGDRAANzonoa6IyBhmkCZ16uzLww== 2023-09-30T00:55:34.730 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.7] 2023-09-30T00:55:34.730 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQADchdl5XsPExAAQnL5Mau9f3C+CXzB0SL5sg== 2023-09-30T00:55:34.730 INFO:teuthology.orchestra.run.smithi069.stdout:[client.0] 2023-09-30T00:55:34.730 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDocRdlAKkRAxAAcDHb5epbhzpT5YqV/uIDiQ== 2023-09-30T00:55:34.730 INFO:teuthology.orchestra.run.smithi069.stdout:[client.1] 2023-09-30T00:55:34.730 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQDocRdlvpkIBhAAhX4XMseGDoRvurO/wSkN5g== 2023-09-30T00:55:34.731 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.y] 2023-09-30T00:55:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDncRdl/l95ORAAti+3wYycUMxqMD27dzvhyA== 2023-09-30T00:55:34.731 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.0] 2023-09-30T00:55:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDqcRdlgpLlJhAAM8Z599+drzwwsaSHh2YT/A== 2023-09-30T00:55:34.731 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.1] 2023-09-30T00:55:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDucRdliUKAABAAZdwBx+vp7jsGTlVcuVJbeg== 2023-09-30T00:55:34.731 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.2] 2023-09-30T00:55:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDxcRdlg4+3BRAAfdk78FFad4T3XIFrD92vgw== 2023-09-30T00:55:34.732 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.3] 2023-09-30T00:55:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQD0cRdlu9cNDBAAuaQgLP5tKVm+kZtP2Kl5/A== 2023-09-30T00:55:34.733 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.x] 2023-09-30T00:55:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDocRdlaGzBABAARNlrZaKMlNvDnAhC3WorMA== 2023-09-30T00:55:34.733 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.4] 2023-09-30T00:55:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQD5cRdlOMUPMhAAPpvfe0HbhG6Nsw97HmjT9A== 2023-09-30T00:55:34.733 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.5] 2023-09-30T00:55:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQD9cRdlBOjuBxAAgYnWh7eHq26T073sPfxUwQ== 2023-09-30T00:55:34.733 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.6] 2023-09-30T00:55:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAAchdlDieGDRAANzonoa6IyBhmkCZ16uzLww== 2023-09-30T00:55:34.733 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.7] 2023-09-30T00:55:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQADchdl5XsPExAAQnL5Mau9f3C+CXzB0SL5sg== 2023-09-30T00:55:34.733 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2023-09-30T00:55:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDocRdlAKkRAxAAcDHb5epbhzpT5YqV/uIDiQ== 2023-09-30T00:55:34.734 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2023-09-30T00:55:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDocRdlvpkIBhAAhX4XMseGDoRvurO/wSkN5g== 2023-09-30T00:55:34.735 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-30T00:55:34.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-30T00:55:34.820 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:34.827 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:34.869 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:34.871 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:34.913 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:34.915 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:35.000 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:35.003 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:35.045 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-30T00:55:35.087 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-30T00:55:35.132 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:35.134 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:35.177 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:35.180 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:35.222 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:35.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:35.311 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:35.313 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T00:55:35.357 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-30T00:55:35.359 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-30T00:55:35.405 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-30T00:55:35.407 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-30T00:55:35.451 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-09-30T00:55:35.451 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-09-30T00:55:35.509 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T00:55:35.563 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-09-30T00:55:35.577 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-09-30T00:55:35.638 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T00:55:35.694 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-09-30T00:55:35.710 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-09-30T00:55:35.727 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T00:55:35.778 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-09-30T00:55:35.792 DEBUG:teuthology.orchestra.run.smithi069:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:35.795 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-30T00:55:35.799 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-09-30T00:55:35.800 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-30T00:55:35.800 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-30T00:55:35.847 INFO:tasks.ceph.mon.a:Started 2023-09-30T00:55:35.848 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-30T00:55:35.848 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-30T00:55:35.850 INFO:tasks.ceph.mon.c:Started 2023-09-30T00:55:35.850 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-30T00:55:35.850 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-30T00:55:35.852 INFO:tasks.ceph.mon.b:Started 2023-09-30T00:55:35.852 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-09-30T00:55:35.853 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-09-30T00:55:35.853 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-09-30T00:55:35.855 INFO:tasks.ceph.mgr.y:Started 2023-09-30T00:55:35.855 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-30T00:55:35.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-30T00:55:35.857 INFO:tasks.ceph.mgr.x:Started 2023-09-30T00:55:35.857 INFO:tasks.ceph:Setting crush tunables to default 2023-09-30T00:55:35.857 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd crush tunables default 2023-09-30T00:55:42.446 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:42.442+0000 7fa5af420000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T00:55:42.655 INFO:teuthology.orchestra.run.smithi069.stderr:adjusted tunables profile to default 2023-09-30T00:55:42.666 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-09-30T00:55:42.666 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:42.666 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-09-30T00:55:42.681 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:42.681 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-09-30T00:55:42.697 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:42.698 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-09-30T00:55:42.708 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-30T00:55:42.709 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-09-30T00:55:42.763 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:42.763 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-09-30T00:55:42.776 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:42.768+0000 7f1d81ed9000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T00:55:42.779 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:42.779 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-09-30T00:55:42.834 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:42.834 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-09-30T00:55:42.849 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-09-30T00:55:42.849 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-09-30T00:55:42.861 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 3644de55-4abe-484a-ab81-37b6248b978c 0 2023-09-30T00:55:43.071 INFO:teuthology.orchestra.run.smithi150.stdout:0 2023-09-30T00:55:43.079 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:43.074+0000 7fa5af420000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T00:55:43.084 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new e81ee49e-57c0-4bd4-81e1-943a53e17e0d 1 2023-09-30T00:55:43.135 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:43.130+0000 7fa5af420000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T00:55:43.250 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:43.242+0000 7fa5af420000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T00:55:43.293 INFO:teuthology.orchestra.run.smithi150.stdout:1 2023-09-30T00:55:43.310 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 95f49092-0cb9-4fcf-8ec8-740061c438c7 2 2023-09-30T00:55:43.441 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:43.432+0000 7f1d81ed9000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T00:55:43.464 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:43.458+0000 7fa5af420000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T00:55:43.498 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:43.492+0000 7f1d81ed9000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T00:55:43.512 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:43.506+0000 7fa5af420000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T00:55:43.517 INFO:teuthology.orchestra.run.smithi150.stdout:2 2023-09-30T00:55:43.531 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 22d0daa1-283e-477d-8088-31922a3ff54f 3 2023-09-30T00:55:43.614 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:43.608+0000 7f1d81ed9000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T00:55:43.687 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:43.682+0000 7fa5af420000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T00:55:43.735 INFO:teuthology.orchestra.run.smithi150.stdout:3 2023-09-30T00:55:43.747 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:43.742+0000 7fa5af420000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T00:55:43.748 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 601a722e-5cd4-46b0-9651-74d99ab73be6 4 2023-09-30T00:55:43.814 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:43.802+0000 7fa5af420000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T00:55:43.842 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:43.832+0000 7f1d81ed9000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T00:55:43.889 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:43.884+0000 7f1d81ed9000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T00:55:43.957 INFO:teuthology.orchestra.run.smithi150.stdout:4 2023-09-30T00:55:43.973 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new c726bca0-dcd7-4041-88b3-6234ddb84477 5 2023-09-30T00:55:44.070 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:44.064+0000 7f1d81ed9000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T00:55:44.135 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:44.128+0000 7f1d81ed9000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T00:55:44.178 INFO:teuthology.orchestra.run.smithi150.stdout:5 2023-09-30T00:55:44.192 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 9dc1bc9b-177d-448c-8917-2c400d1d3ca4 6 2023-09-30T00:55:44.197 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:44.192+0000 7f1d81ed9000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T00:55:44.201 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:44.194+0000 7fa5af420000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T00:55:44.311 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:44.306+0000 7fa5af420000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T00:55:44.388 INFO:teuthology.orchestra.run.smithi150.stdout:6 2023-09-30T00:55:44.401 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 45e5f524-aa7c-46a7-80a1-bfbeac544530 7 2023-09-30T00:55:44.599 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:44.592+0000 7f1d81ed9000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T00:55:44.601 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:44.594+0000 7fa5af420000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T00:55:44.652 INFO:teuthology.orchestra.run.smithi150.stdout:7 2023-09-30T00:55:44.664 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-30T00:55:44.664 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-30T00:55:44.668 INFO:tasks.ceph.osd.0:Started 2023-09-30T00:55:44.668 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-30T00:55:44.668 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-30T00:55:44.671 INFO:tasks.ceph.osd.1:Started 2023-09-30T00:55:44.671 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-30T00:55:44.671 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-30T00:55:44.674 INFO:tasks.ceph.osd.2:Started 2023-09-30T00:55:44.674 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-30T00:55:44.674 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-30T00:55:44.679 INFO:tasks.ceph.osd.3:Started 2023-09-30T00:55:44.679 INFO:tasks.ceph.osd.4:Restarting daemon 2023-09-30T00:55:44.679 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-09-30T00:55:44.682 INFO:tasks.ceph.osd.4:Started 2023-09-30T00:55:44.682 INFO:tasks.ceph.osd.5:Restarting daemon 2023-09-30T00:55:44.683 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-09-30T00:55:44.685 INFO:tasks.ceph.osd.5:Started 2023-09-30T00:55:44.685 INFO:tasks.ceph.osd.6:Restarting daemon 2023-09-30T00:55:44.685 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-09-30T00:55:44.688 INFO:tasks.ceph.osd.6:Started 2023-09-30T00:55:44.688 INFO:tasks.ceph.osd.7:Restarting daemon 2023-09-30T00:55:44.689 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-09-30T00:55:44.691 INFO:tasks.ceph.osd.7:Started 2023-09-30T00:55:44.691 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T00:55:44.732 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:44.724+0000 7f1d81ed9000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T00:55:44.782 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:44.774+0000 7fa5af420000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T00:55:44.865 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:44.858+0000 7fa5af420000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T00:55:44.882 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:44.882 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":10,"fsid":"a4f1df1b-c238-45bb-b595-c54d4bb08739","created":"2023-09-30T00:55:42.211215+0000","modified":"2023-09-30T00:55:44.646942+0000","last_up_change":"0.000000","last_in_change":"2023-09-30T00:55:44.646942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3644de55-4abe-484a-ab81-37b6248b978c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e81ee49e-57c0-4bd4-81e1-943a53e17e0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"95f49092-0cb9-4fcf-8ec8-740061c438c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"22d0daa1-283e-477d-8088-31922a3ff54f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"601a722e-5cd4-46b0-9651-74d99ab73be6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c726bca0-dcd7-4041-88b3-6234ddb84477","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9dc1bc9b-177d-448c-8917-2c400d1d3ca4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"45e5f524-aa7c-46a7-80a1-bfbeac544530","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T00:55:44.892 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-09-30T00:55:44.892 INFO:tasks.ceph:Waiting for OSDs to come up 2023-09-30T00:55:44.920 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:44.914+0000 7fa5af420000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T00:55:44.981 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:44.974+0000 7fa5af420000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T00:55:45.032 INFO:tasks.ceph.osd.3.smithi069.stderr:2023-09-30T00:55:45.026+0000 7fc32ef003c0 -1 Falling back to public interface 2023-09-30T00:55:45.037 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:45.028+0000 7f1d81ed9000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T00:55:45.046 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-09-30T00:55:45.040+0000 7f5b937653c0 -1 Falling back to public interface 2023-09-30T00:55:45.056 INFO:tasks.ceph.osd.2.smithi069.stderr:2023-09-30T00:55:45.050+0000 7ff39cca83c0 -1 Falling back to public interface 2023-09-30T00:55:45.065 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-09-30T00:55:45.060+0000 7f2ec198c3c0 -1 Falling back to public interface 2023-09-30T00:55:45.191 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:45.184+0000 7f1d81ed9000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T00:55:45.257 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:45.252+0000 7f1d81ed9000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T00:55:45.264 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:45.258+0000 7fa5af420000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T00:55:45.284 INFO:tasks.ceph.osd.0.smithi069.stderr:2023-09-30T00:55:45.278+0000 7f4ea135b3c0 -1 Falling back to public interface 2023-09-30T00:55:45.295 INFO:tasks.ceph.osd.1.smithi069.stderr:2023-09-30T00:55:45.290+0000 7f1c3ab013c0 -1 Falling back to public interface 2023-09-30T00:55:45.309 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:45.304+0000 7f1d81ed9000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T00:55:45.314 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-09-30T00:55:45.308+0000 7f9d8e8423c0 -1 Falling back to public interface 2023-09-30T00:55:45.326 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-09-30T00:55:45.320+0000 7f9d6880f3c0 -1 Falling back to public interface 2023-09-30T00:55:45.327 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:45.322+0000 7fa5af420000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T00:55:45.362 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:45.356+0000 7f1d81ed9000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T00:55:45.396 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:45.390+0000 7fa5af420000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T00:55:45.568 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:45.562+0000 7fa5af420000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T00:55:45.642 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:45.634+0000 7fa5af420000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T00:55:45.667 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:45.660+0000 7f1d81ed9000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T00:55:45.698 DEBUG:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-30T00:55:45.721 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:45.716+0000 7f1d81ed9000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T00:55:45.781 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:45.776+0000 7f1d81ed9000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T00:55:45.878 INFO:teuthology.misc.health.smithi069.stdout: 2023-09-30T00:55:45.878 INFO:teuthology.misc.health.smithi069.stdout:{"epoch":10,"fsid":"a4f1df1b-c238-45bb-b595-c54d4bb08739","created":"2023-09-30T00:55:42.211215+0000","modified":"2023-09-30T00:55:44.646942+0000","last_up_change":"0.000000","last_in_change":"2023-09-30T00:55:44.646942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3644de55-4abe-484a-ab81-37b6248b978c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e81ee49e-57c0-4bd4-81e1-943a53e17e0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"95f49092-0cb9-4fcf-8ec8-740061c438c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"22d0daa1-283e-477d-8088-31922a3ff54f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"601a722e-5cd4-46b0-9651-74d99ab73be6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c726bca0-dcd7-4041-88b3-6234ddb84477","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9dc1bc9b-177d-448c-8917-2c400d1d3ca4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"45e5f524-aa7c-46a7-80a1-bfbeac544530","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T00:55:45.889 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-09-30T00:55:45.940 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:45.932+0000 7f1d81ed9000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T00:55:46.007 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:46.000+0000 7f1d81ed9000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T00:55:46.365 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:46.358+0000 7fa5af420000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T00:55:46.422 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:46.414+0000 7fa58cd01700 -1 log_channel(cluster) log [ERR] : Unhandled exception from module 'pg_autoscaler' while running on mgr.y: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-30T00:55:46.422 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:46.414+0000 7fa58cd01700 -1 pg_autoscaler.serve: 2023-09-30T00:55:46.423 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:46.414+0000 7fa58cd01700 -1 Traceback (most recent call last): 2023-09-30T00:55:46.424 INFO:tasks.ceph.mgr.y.smithi069.stderr: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 325, in serve 2023-09-30T00:55:46.424 INFO:tasks.ceph.mgr.y.smithi069.stderr: self._maybe_adjust() 2023-09-30T00:55:46.424 INFO:tasks.ceph.mgr.y.smithi069.stderr: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 682, in _maybe_adjust 2023-09-30T00:55:46.424 INFO:tasks.ceph.mgr.y.smithi069.stderr: if self.noautoscale: 2023-09-30T00:55:46.424 INFO:tasks.ceph.mgr.y.smithi069.stderr:AttributeError: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-30T00:55:46.424 INFO:tasks.ceph.mgr.y.smithi069.stderr: 2023-09-30T00:55:46.717 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-09-30T00:55:46.712+0000 7f1d81ed9000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T00:55:47.410 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:47.406+0000 7fa598518700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-30T00:55:47.412 INFO:tasks.ceph.mgr.y.smithi069.stderr:2023-09-30T00:55:47.406+0000 7fa598518700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-30T00:55:48.712 INFO:tasks.ceph.osd.2.smithi069.stderr:2023-09-30T00:55:48.706+0000 7ff39cca83c0 -1 osd.2 0 log_to_monitors true 2023-09-30T00:55:48.721 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-09-30T00:55:48.716+0000 7f5b937653c0 -1 osd.4 0 log_to_monitors true 2023-09-30T00:55:48.854 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-09-30T00:55:48.848+0000 7f2ec198c3c0 -1 osd.5 0 log_to_monitors true 2023-09-30T00:55:48.940 INFO:tasks.ceph.osd.3.smithi069.stderr:2023-09-30T00:55:48.934+0000 7fc32ef003c0 -1 osd.3 0 log_to_monitors true 2023-09-30T00:55:48.960 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-09-30T00:55:48.956+0000 7f9d6880f3c0 -1 osd.6 0 log_to_monitors true 2023-09-30T00:55:49.102 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-09-30T00:55:49.096+0000 7f9d8e8423c0 -1 osd.7 0 log_to_monitors true 2023-09-30T00:55:49.111 INFO:tasks.ceph.osd.1.smithi069.stderr:2023-09-30T00:55:49.106+0000 7f1c3ab013c0 -1 osd.1 0 log_to_monitors true 2023-09-30T00:55:49.131 INFO:tasks.ceph.osd.0.smithi069.stderr:2023-09-30T00:55:49.126+0000 7f4ea135b3c0 -1 osd.0 0 log_to_monitors true 2023-09-30T00:55:50.278 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-09-30T00:55:50.272+0000 7f5b8ef05700 -1 osd.4 0 waiting for initial osdmap 2023-09-30T00:55:50.279 INFO:tasks.ceph.osd.3.smithi069.stderr:2023-09-30T00:55:50.274+0000 7fc32a6a0700 -1 osd.3 0 waiting for initial osdmap 2023-09-30T00:55:50.281 INFO:tasks.ceph.osd.0.smithi069.stderr:2023-09-30T00:55:50.274+0000 7f4e9cafb700 -1 osd.0 0 waiting for initial osdmap 2023-09-30T00:55:50.282 INFO:tasks.ceph.osd.1.smithi069.stderr:2023-09-30T00:55:50.278+0000 7f1c37ac6700 -1 osd.1 0 waiting for initial osdmap 2023-09-30T00:55:50.282 INFO:tasks.ceph.osd.2.smithi069.stderr:2023-09-30T00:55:50.278+0000 7ff399c6d700 -1 osd.2 0 waiting for initial osdmap 2023-09-30T00:55:50.282 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-09-30T00:55:50.276+0000 7f9d89fe2700 -1 osd.7 0 waiting for initial osdmap 2023-09-30T00:55:50.283 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-09-30T00:55:50.276+0000 7f9d657d4700 -1 osd.6 0 waiting for initial osdmap 2023-09-30T00:55:50.283 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-09-30T00:55:50.276+0000 7f2ebd12c700 -1 osd.5 0 waiting for initial osdmap 2023-09-30T00:55:50.288 INFO:tasks.ceph.osd.3.smithi069.stderr:2023-09-30T00:55:50.282+0000 7fc324446700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T00:55:50.289 INFO:tasks.ceph.osd.1.smithi069.stderr:2023-09-30T00:55:50.282+0000 7f1c2f846700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T00:55:50.289 INFO:tasks.ceph.osd.0.smithi069.stderr:2023-09-30T00:55:50.282+0000 7f4e960a0700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T00:55:50.290 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-09-30T00:55:50.284+0000 7f9d5dd55700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T00:55:50.291 INFO:tasks.ceph.osd.2.smithi069.stderr:2023-09-30T00:55:50.286+0000 7ff3921ee700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T00:55:50.293 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-09-30T00:55:50.288+0000 7f5b88cab700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T00:55:50.294 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-09-30T00:55:50.288+0000 7f2eb6ed2700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T00:55:50.294 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-09-30T00:55:50.288+0000 7f9d83587700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T00:55:52.286 INFO:tasks.ceph.mon.a.smithi069.stderr:2023-09-30T00:55:52.282+0000 7fa1fbbd5700 -1 log_channel(cluster) log [ERR] : Health check failed: Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale' (MGR_MODULE_ERROR) 2023-09-30T00:55:52.695 DEBUG:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-30T00:55:52.986 INFO:teuthology.misc.health.smithi069.stdout: 2023-09-30T00:55:52.986 INFO:teuthology.misc.health.smithi069.stdout:{"epoch":14,"fsid":"a4f1df1b-c238-45bb-b595-c54d4bb08739","created":"2023-09-30T00:55:42.211215+0000","modified":"2023-09-30T00:55:52.288877+0000","last_up_change":"2023-09-30T00:55:51.275589+0000","last_in_change":"2023-09-30T00:55:44.646942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3644de55-4abe-484a-ab81-37b6248b978c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":31243},{"type":"v1","addr":"172.21.15.69:6817","nonce":31243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":31243},{"type":"v1","addr":"172.21.15.69:6819","nonce":31243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":31243},{"type":"v1","addr":"172.21.15.69:6823","nonce":31243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":31243},{"type":"v1","addr":"172.21.15.69:6821","nonce":31243}]},"public_addr":"172.21.15.69:6817/31243","cluster_addr":"172.21.15.69:6819/31243","heartbeat_back_addr":"172.21.15.69:6823/31243","heartbeat_front_addr":"172.21.15.69:6821/31243","state":["exists","up"]},{"osd":1,"uuid":"e81ee49e-57c0-4bd4-81e1-943a53e17e0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":31242},{"type":"v1","addr":"172.21.15.69:6825","nonce":31242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":31242},{"type":"v1","addr":"172.21.15.69:6827","nonce":31242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":31242},{"type":"v1","addr":"172.21.15.69:6831","nonce":31242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":31242},{"type":"v1","addr":"172.21.15.69:6829","nonce":31242}]},"public_addr":"172.21.15.69:6825/31242","cluster_addr":"172.21.15.69:6827/31242","heartbeat_back_addr":"172.21.15.69:6831/31242","heartbeat_front_addr":"172.21.15.69:6829/31242","state":["exists","up"]},{"osd":2,"uuid":"95f49092-0cb9-4fcf-8ec8-740061c438c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":31245},{"type":"v1","addr":"172.21.15.69:6809","nonce":31245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":31245},{"type":"v1","addr":"172.21.15.69:6811","nonce":31245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":31245},{"type":"v1","addr":"172.21.15.69:6815","nonce":31245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":31245},{"type":"v1","addr":"172.21.15.69:6813","nonce":31245}]},"public_addr":"172.21.15.69:6809/31245","cluster_addr":"172.21.15.69:6811/31245","heartbeat_back_addr":"172.21.15.69:6815/31245","heartbeat_front_addr":"172.21.15.69:6813/31245","state":["exists","up"]},{"osd":3,"uuid":"22d0daa1-283e-477d-8088-31922a3ff54f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":31241},{"type":"v1","addr":"172.21.15.69:6801","nonce":31241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":31241},{"type":"v1","addr":"172.21.15.69:6803","nonce":31241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":31241},{"type":"v1","addr":"172.21.15.69:6807","nonce":31241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":31241},{"type":"v1","addr":"172.21.15.69:6805","nonce":31241}]},"public_addr":"172.21.15.69:6801/31241","cluster_addr":"172.21.15.69:6803/31241","heartbeat_back_addr":"172.21.15.69:6807/31241","heartbeat_front_addr":"172.21.15.69:6805/31241","state":["exists","up"]},{"osd":4,"uuid":"601a722e-5cd4-46b0-9651-74d99ab73be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":31269},{"type":"v1","addr":"172.21.15.150:6801","nonce":31269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":31269},{"type":"v1","addr":"172.21.15.150:6803","nonce":31269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":31269},{"type":"v1","addr":"172.21.15.150:6807","nonce":31269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":31269},{"type":"v1","addr":"172.21.15.150:6805","nonce":31269}]},"public_addr":"172.21.15.150:6801/31269","cluster_addr":"172.21.15.150:6803/31269","heartbeat_back_addr":"172.21.15.150:6807/31269","heartbeat_front_addr":"172.21.15.150:6805/31269","state":["exists","up"]},{"osd":5,"uuid":"c726bca0-dcd7-4041-88b3-6234ddb84477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":31270},{"type":"v1","addr":"172.21.15.150:6809","nonce":31270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":31270},{"type":"v1","addr":"172.21.15.150:6811","nonce":31270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":31270},{"type":"v1","addr":"172.21.15.150:6815","nonce":31270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":31270},{"type":"v1","addr":"172.21.15.150:6813","nonce":31270}]},"public_addr":"172.21.15.150:6809/31270","cluster_addr":"172.21.15.150:6811/31270","heartbeat_back_addr":"172.21.15.150:6815/31270","heartbeat_front_addr":"172.21.15.150:6813/31270","state":["exists","up"]},{"osd":6,"uuid":"9dc1bc9b-177d-448c-8917-2c400d1d3ca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":31271},{"type":"v1","addr":"172.21.15.150:6825","nonce":31271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":31271},{"type":"v1","addr":"172.21.15.150:6827","nonce":31271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":31271},{"type":"v1","addr":"172.21.15.150:6831","nonce":31271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":31271},{"type":"v1","addr":"172.21.15.150:6829","nonce":31271}]},"public_addr":"172.21.15.150:6825/31271","cluster_addr":"172.21.15.150:6827/31271","heartbeat_back_addr":"172.21.15.150:6831/31271","heartbeat_front_addr":"172.21.15.150:6829/31271","state":["exists","up"]},{"osd":7,"uuid":"45e5f524-aa7c-46a7-80a1-bfbeac544530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":31268},{"type":"v1","addr":"172.21.15.150:6817","nonce":31268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":31268},{"type":"v1","addr":"172.21.15.150:6819","nonce":31268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":31268},{"type":"v1","addr":"172.21.15.150:6823","nonce":31268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":31268},{"type":"v1","addr":"172.21.15.150:6821","nonce":31268}]},"public_addr":"172.21.15.150:6817/31268","cluster_addr":"172.21.15.150:6819/31268","heartbeat_back_addr":"172.21.15.150:6823/31268","heartbeat_front_addr":"172.21.15.150:6821/31268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:50.170257+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:50.141575+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.761411+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.971165+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.705679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.830153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.989276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:50.063003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T00:55:52.998 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-09-30T00:55:52.999 INFO:tasks.ceph:Creating RBD pool 2023-09-30T00:55:52.999 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd pool create rbd 8 2023-09-30T00:55:54.333 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'rbd' created 2023-09-30T00:55:54.345 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-09-30T00:55:55.341 INFO:teuthology.orchestra.run.smithi069.stderr:enabled application 'rbd' on pool 'rbd' 2023-09-30T00:55:55.355 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-09-30T00:55:55.355 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T00:55:55.356 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-30T00:55:55.356 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-30T00:55:55.356 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-09-30T00:55:55.688 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:55.701 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":4,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":31077},{"type":"v1","addr":"172.21.15.69:6833","nonce":31077}]},"active_addr":"172.21.15.69:6833/31077","active_change":"2023-09-30T00:55:46.383794+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3927954175}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2507503884}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3750827590}]}]} 2023-09-30T00:55:55.707 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-30T00:55:55.707 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T00:55:55.707 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T00:55:55.990 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:55.991 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"fsid":"a4f1df1b-c238-45bb-b595-c54d4bb08739","created":"2023-09-30T00:55:42.211215+0000","modified":"2023-09-30T00:55:55.320421+0000","last_up_change":"2023-09-30T00:55:51.275589+0000","last_in_change":"2023-09-30T00:55:44.646942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T00:55:52.460922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T00:55:53.353727+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3644de55-4abe-484a-ab81-37b6248b978c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":31243},{"type":"v1","addr":"172.21.15.69:6817","nonce":31243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":31243},{"type":"v1","addr":"172.21.15.69:6819","nonce":31243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":31243},{"type":"v1","addr":"172.21.15.69:6823","nonce":31243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":31243},{"type":"v1","addr":"172.21.15.69:6821","nonce":31243}]},"public_addr":"172.21.15.69:6817/31243","cluster_addr":"172.21.15.69:6819/31243","heartbeat_back_addr":"172.21.15.69:6823/31243","heartbeat_front_addr":"172.21.15.69:6821/31243","state":["exists","up"]},{"osd":1,"uuid":"e81ee49e-57c0-4bd4-81e1-943a53e17e0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":31242},{"type":"v1","addr":"172.21.15.69:6825","nonce":31242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":31242},{"type":"v1","addr":"172.21.15.69:6827","nonce":31242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":31242},{"type":"v1","addr":"172.21.15.69:6831","nonce":31242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":31242},{"type":"v1","addr":"172.21.15.69:6829","nonce":31242}]},"public_addr":"172.21.15.69:6825/31242","cluster_addr":"172.21.15.69:6827/31242","heartbeat_back_addr":"172.21.15.69:6831/31242","heartbeat_front_addr":"172.21.15.69:6829/31242","state":["exists","up"]},{"osd":2,"uuid":"95f49092-0cb9-4fcf-8ec8-740061c438c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":31245},{"type":"v1","addr":"172.21.15.69:6809","nonce":31245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":31245},{"type":"v1","addr":"172.21.15.69:6811","nonce":31245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":31245},{"type":"v1","addr":"172.21.15.69:6815","nonce":31245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":31245},{"type":"v1","addr":"172.21.15.69:6813","nonce":31245}]},"public_addr":"172.21.15.69:6809/31245","cluster_addr":"172.21.15.69:6811/31245","heartbeat_back_addr":"172.21.15.69:6815/31245","heartbeat_front_addr":"172.21.15.69:6813/31245","state":["exists","up"]},{"osd":3,"uuid":"22d0daa1-283e-477d-8088-31922a3ff54f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":31241},{"type":"v1","addr":"172.21.15.69:6801","nonce":31241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":31241},{"type":"v1","addr":"172.21.15.69:6803","nonce":31241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":31241},{"type":"v1","addr":"172.21.15.69:6807","nonce":31241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":31241},{"type":"v1","addr":"172.21.15.69:6805","nonce":31241}]},"public_addr":"172.21.15.69:6801/31241","cluster_addr":"172.21.15.69:6803/31241","heartbeat_back_addr":"172.21.15.69:6807/31241","heartbeat_front_addr":"172.21.15.69:6805/31241","state":["exists","up"]},{"osd":4,"uuid":"601a722e-5cd4-46b0-9651-74d99ab73be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":31269},{"type":"v1","addr":"172.21.15.150:6801","nonce":31269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":31269},{"type":"v1","addr":"172.21.15.150:6803","nonce":31269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":31269},{"type":"v1","addr":"172.21.15.150:6807","nonce":31269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":31269},{"type":"v1","addr":"172.21.15.150:6805","nonce":31269}]},"public_addr":"172.21.15.150:6801/31269","cluster_addr":"172.21.15.150:6803/31269","heartbeat_back_addr":"172.21.15.150:6807/31269","heartbeat_front_addr":"172.21.15.150:6805/31269","state":["exists","up"]},{"osd":5,"uuid":"c726bca0-dcd7-4041-88b3-6234ddb84477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":31270},{"type":"v1","addr":"172.21.15.150:6809","nonce":31270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":31270},{"type":"v1","addr":"172.21.15.150:6811","nonce":31270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":31270},{"type":"v1","addr":"172.21.15.150:6815","nonce":31270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":31270},{"type":"v1","addr":"172.21.15.150:6813","nonce":31270}]},"public_addr":"172.21.15.150:6809/31270","cluster_addr":"172.21.15.150:6811/31270","heartbeat_back_addr":"172.21.15.150:6815/31270","heartbeat_front_addr":"172.21.15.150:6813/31270","state":["exists","up"]},{"osd":6,"uuid":"9dc1bc9b-177d-448c-8917-2c400d1d3ca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":31271},{"type":"v1","addr":"172.21.15.150:6825","nonce":31271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":31271},{"type":"v1","addr":"172.21.15.150:6827","nonce":31271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":31271},{"type":"v1","addr":"172.21.15.150:6831","nonce":31271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":31271},{"type":"v1","addr":"172.21.15.150:6829","nonce":31271}]},"public_addr":"172.21.15.150:6825/31271","cluster_addr":"172.21.15.150:6827/31271","heartbeat_back_addr":"172.21.15.150:6831/31271","heartbeat_front_addr":"172.21.15.150:6829/31271","state":["exists","up"]},{"osd":7,"uuid":"45e5f524-aa7c-46a7-80a1-bfbeac544530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":31268},{"type":"v1","addr":"172.21.15.150:6817","nonce":31268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":31268},{"type":"v1","addr":"172.21.15.150:6819","nonce":31268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":31268},{"type":"v1","addr":"172.21.15.150:6823","nonce":31268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":31268},{"type":"v1","addr":"172.21.15.150:6821","nonce":31268}]},"public_addr":"172.21.15.150:6817/31268","cluster_addr":"172.21.15.150:6819/31268","heartbeat_back_addr":"172.21.15.150:6823/31268","heartbeat_front_addr":"172.21.15.150:6821/31268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:50.170257+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:50.141575+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.761411+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.971165+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.705679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.830153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.989276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:50.063003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T00:55:56.010 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T00:55:56.011 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T00:55:56.301 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:55:56.301 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"fsid":"a4f1df1b-c238-45bb-b595-c54d4bb08739","created":"2023-09-30T00:55:42.211215+0000","modified":"2023-09-30T00:55:55.320421+0000","last_up_change":"2023-09-30T00:55:51.275589+0000","last_in_change":"2023-09-30T00:55:44.646942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T00:55:52.460922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T00:55:53.353727+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3644de55-4abe-484a-ab81-37b6248b978c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":31243},{"type":"v1","addr":"172.21.15.69:6817","nonce":31243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":31243},{"type":"v1","addr":"172.21.15.69:6819","nonce":31243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":31243},{"type":"v1","addr":"172.21.15.69:6823","nonce":31243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":31243},{"type":"v1","addr":"172.21.15.69:6821","nonce":31243}]},"public_addr":"172.21.15.69:6817/31243","cluster_addr":"172.21.15.69:6819/31243","heartbeat_back_addr":"172.21.15.69:6823/31243","heartbeat_front_addr":"172.21.15.69:6821/31243","state":["exists","up"]},{"osd":1,"uuid":"e81ee49e-57c0-4bd4-81e1-943a53e17e0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":31242},{"type":"v1","addr":"172.21.15.69:6825","nonce":31242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":31242},{"type":"v1","addr":"172.21.15.69:6827","nonce":31242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":31242},{"type":"v1","addr":"172.21.15.69:6831","nonce":31242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":31242},{"type":"v1","addr":"172.21.15.69:6829","nonce":31242}]},"public_addr":"172.21.15.69:6825/31242","cluster_addr":"172.21.15.69:6827/31242","heartbeat_back_addr":"172.21.15.69:6831/31242","heartbeat_front_addr":"172.21.15.69:6829/31242","state":["exists","up"]},{"osd":2,"uuid":"95f49092-0cb9-4fcf-8ec8-740061c438c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":31245},{"type":"v1","addr":"172.21.15.69:6809","nonce":31245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":31245},{"type":"v1","addr":"172.21.15.69:6811","nonce":31245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":31245},{"type":"v1","addr":"172.21.15.69:6815","nonce":31245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":31245},{"type":"v1","addr":"172.21.15.69:6813","nonce":31245}]},"public_addr":"172.21.15.69:6809/31245","cluster_addr":"172.21.15.69:6811/31245","heartbeat_back_addr":"172.21.15.69:6815/31245","heartbeat_front_addr":"172.21.15.69:6813/31245","state":["exists","up"]},{"osd":3,"uuid":"22d0daa1-283e-477d-8088-31922a3ff54f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":31241},{"type":"v1","addr":"172.21.15.69:6801","nonce":31241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":31241},{"type":"v1","addr":"172.21.15.69:6803","nonce":31241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":31241},{"type":"v1","addr":"172.21.15.69:6807","nonce":31241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":31241},{"type":"v1","addr":"172.21.15.69:6805","nonce":31241}]},"public_addr":"172.21.15.69:6801/31241","cluster_addr":"172.21.15.69:6803/31241","heartbeat_back_addr":"172.21.15.69:6807/31241","heartbeat_front_addr":"172.21.15.69:6805/31241","state":["exists","up"]},{"osd":4,"uuid":"601a722e-5cd4-46b0-9651-74d99ab73be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":31269},{"type":"v1","addr":"172.21.15.150:6801","nonce":31269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":31269},{"type":"v1","addr":"172.21.15.150:6803","nonce":31269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":31269},{"type":"v1","addr":"172.21.15.150:6807","nonce":31269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":31269},{"type":"v1","addr":"172.21.15.150:6805","nonce":31269}]},"public_addr":"172.21.15.150:6801/31269","cluster_addr":"172.21.15.150:6803/31269","heartbeat_back_addr":"172.21.15.150:6807/31269","heartbeat_front_addr":"172.21.15.150:6805/31269","state":["exists","up"]},{"osd":5,"uuid":"c726bca0-dcd7-4041-88b3-6234ddb84477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":31270},{"type":"v1","addr":"172.21.15.150:6809","nonce":31270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":31270},{"type":"v1","addr":"172.21.15.150:6811","nonce":31270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":31270},{"type":"v1","addr":"172.21.15.150:6815","nonce":31270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":31270},{"type":"v1","addr":"172.21.15.150:6813","nonce":31270}]},"public_addr":"172.21.15.150:6809/31270","cluster_addr":"172.21.15.150:6811/31270","heartbeat_back_addr":"172.21.15.150:6815/31270","heartbeat_front_addr":"172.21.15.150:6813/31270","state":["exists","up"]},{"osd":6,"uuid":"9dc1bc9b-177d-448c-8917-2c400d1d3ca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":31271},{"type":"v1","addr":"172.21.15.150:6825","nonce":31271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":31271},{"type":"v1","addr":"172.21.15.150:6827","nonce":31271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":31271},{"type":"v1","addr":"172.21.15.150:6831","nonce":31271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":31271},{"type":"v1","addr":"172.21.15.150:6829","nonce":31271}]},"public_addr":"172.21.15.150:6825/31271","cluster_addr":"172.21.15.150:6827/31271","heartbeat_back_addr":"172.21.15.150:6831/31271","heartbeat_front_addr":"172.21.15.150:6829/31271","state":["exists","up"]},{"osd":7,"uuid":"45e5f524-aa7c-46a7-80a1-bfbeac544530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":31268},{"type":"v1","addr":"172.21.15.150:6817","nonce":31268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":31268},{"type":"v1","addr":"172.21.15.150:6819","nonce":31268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":31268},{"type":"v1","addr":"172.21.15.150:6823","nonce":31268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":31268},{"type":"v1","addr":"172.21.15.150:6821","nonce":31268}]},"public_addr":"172.21.15.150:6817/31268","cluster_addr":"172.21.15.150:6819/31268","heartbeat_back_addr":"172.21.15.150:6823/31268","heartbeat_front_addr":"172.21.15.150:6821/31268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:50.170257+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:50.141575+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.761411+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.971165+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.705679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.830153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:49.989276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:55:50.063003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T00:55:56.319 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-30T00:55:56.320 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-30T00:55:56.320 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-30T00:55:56.321 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-30T00:55:56.321 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-09-30T00:55:56.321 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-09-30T00:55:56.322 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-09-30T00:55:56.322 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-09-30T00:55:56.528 INFO:teuthology.orchestra.run.smithi069.stdout:55834574852 2023-09-30T00:55:56.529 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-30T00:55:56.532 INFO:teuthology.orchestra.run.smithi069.stdout:55834574850 2023-09-30T00:55:56.532 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T00:55:56.533 INFO:teuthology.orchestra.run.smithi069.stdout:55834574851 2023-09-30T00:55:56.534 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-30T00:55:56.617 INFO:teuthology.orchestra.run.smithi069.stdout:55834574850 2023-09-30T00:55:56.618 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T00:55:56.625 INFO:teuthology.orchestra.run.smithi069.stdout:55834574850 2023-09-30T00:55:56.625 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T00:55:56.629 INFO:teuthology.orchestra.run.smithi069.stdout:55834574851 2023-09-30T00:55:56.630 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-30T00:55:56.640 INFO:teuthology.orchestra.run.smithi069.stdout:55834574851 2023-09-30T00:55:56.641 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T00:55:56.641 INFO:teuthology.orchestra.run.smithi069.stdout:55834574851 2023-09-30T00:55:56.642 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-30T00:55:56.969 INFO:teuthology.orchestra.run.smithi069.stdout:55834574849 2023-09-30T00:55:56.978 INFO:teuthology.orchestra.run.smithi069.stdout:55834574850 2023-09-30T00:55:56.990 INFO:teuthology.orchestra.run.smithi069.stdout:55834574849 2023-09-30T00:55:56.994 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.1 2023-09-30T00:55:57.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.6 2023-09-30T00:55:57.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.5 2023-09-30T00:55:57.097 INFO:teuthology.orchestra.run.smithi069.stdout:55834574849 2023-09-30T00:55:57.100 INFO:teuthology.orchestra.run.smithi069.stdout:55834574849 2023-09-30T00:55:57.105 INFO:teuthology.orchestra.run.smithi069.stdout:55834574849 2023-09-30T00:55:57.112 INFO:teuthology.orchestra.run.smithi069.stdout:55834574849 2023-09-30T00:55:57.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.0 2023-09-30T00:55:57.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.2 2023-09-30T00:55:57.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.7 2023-09-30T00:55:57.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.3 2023-09-30T00:55:57.160 INFO:teuthology.orchestra.run.smithi069.stdout:55834574850 2023-09-30T00:55:57.173 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2023-09-30T00:55:57.995 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T00:55:58.002 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-30T00:55:58.007 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-30T00:55:58.115 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T00:55:58.118 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T00:55:58.119 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-30T00:55:58.127 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T00:55:58.174 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-30T00:55:58.410 INFO:teuthology.orchestra.run.smithi069.stdout:55834574851 2023-09-30T00:55:58.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.6 2023-09-30T00:55:58.438 INFO:teuthology.orchestra.run.smithi069.stdout:55834574849 2023-09-30T00:55:58.442 INFO:teuthology.orchestra.run.smithi069.stdout:55834574850 2023-09-30T00:55:58.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.1 2023-09-30T00:55:58.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2023-09-30T00:55:58.518 INFO:teuthology.orchestra.run.smithi069.stdout:55834574850 2023-09-30T00:55:58.528 INFO:teuthology.orchestra.run.smithi069.stdout:55834574850 2023-09-30T00:55:58.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2023-09-30T00:55:58.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2023-09-30T00:55:58.549 INFO:teuthology.orchestra.run.smithi069.stdout:55834574849 2023-09-30T00:55:58.552 INFO:teuthology.orchestra.run.smithi069.stdout:55834574849 2023-09-30T00:55:58.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.3 2023-09-30T00:55:58.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.2 2023-09-30T00:55:58.595 INFO:teuthology.orchestra.run.smithi069.stdout:55834574850 2023-09-30T00:55:58.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2023-09-30T00:55:59.433 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-30T00:55:59.489 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T00:55:59.489 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-30T00:55:59.531 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-30T00:55:59.540 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T00:55:59.565 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T00:55:59.567 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T00:55:59.608 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-30T00:55:59.880 INFO:teuthology.orchestra.run.smithi069.stdout:55834574851 2023-09-30T00:55:59.900 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.1 2023-09-30T00:55:59.900 DEBUG:teuthology.parallel:result is None 2023-09-30T00:55:59.926 INFO:teuthology.orchestra.run.smithi069.stdout:55834574852 2023-09-30T00:55:59.943 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2023-09-30T00:55:59.944 DEBUG:teuthology.parallel:result is None 2023-09-30T00:55:59.944 INFO:teuthology.orchestra.run.smithi069.stdout:55834574851 2023-09-30T00:55:59.956 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2023-09-30T00:55:59.956 DEBUG:teuthology.parallel:result is None 2023-09-30T00:55:59.986 INFO:teuthology.orchestra.run.smithi069.stdout:55834574851 2023-09-30T00:55:59.988 INFO:teuthology.orchestra.run.smithi069.stdout:55834574851 2023-09-30T00:55:59.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.2 2023-09-30T00:55:59.999 DEBUG:teuthology.parallel:result is None 2023-09-30T00:56:00.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2023-09-30T00:56:00.007 DEBUG:teuthology.parallel:result is None 2023-09-30T00:56:00.019 INFO:teuthology.orchestra.run.smithi069.stdout:55834574851 2023-09-30T00:56:00.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2023-09-30T00:56:00.031 DEBUG:teuthology.parallel:result is None 2023-09-30T00:56:00.050 INFO:teuthology.orchestra.run.smithi069.stdout:55834574851 2023-09-30T00:56:00.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.3 2023-09-30T00:56:00.067 DEBUG:teuthology.parallel:result is None 2023-09-30T00:56:00.079 INFO:teuthology.orchestra.run.smithi069.stdout:55834574852 2023-09-30T00:56:00.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.4 2023-09-30T00:56:00.091 DEBUG:teuthology.parallel:result is None 2023-09-30T00:56:00.092 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T00:56:00.092 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T00:56:00.364 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:00.366 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-09-30T00:56:00.376 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2023-09-30T00:55:58.399049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":95,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":42392,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":40768,"kb_avail":754932328,"statfs":{"total":773094113280,"available":773050703872,"internally_reserved":0,"allocated":1531904,"data_stored":1707804,"data_compressed":69833,"data_compressed_allocated":94208,"data_compressed_original":1228800,"omap_allocated":0,"internal_metadata":41746432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.052595"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.344142+0000","last_change":"2023-09-30T00:55:55.344142+0000","last_active":"2023-09-30T00:55:55.344142+0000","last_peered":"2023-09-30T00:55:55.344142+0000","last_clean":"2023-09-30T00:55:55.344142+0000","last_became_active":"2023-09-30T00:55:55.343702+0000","last_became_peered":"2023-09-30T00:55:55.343702+0000","last_unstale":"2023-09-30T00:55:55.344142+0000","last_undegraded":"2023-09-30T00:55:55.344142+0000","last_fullsized":"2023-09-30T00:55:55.344142+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:10.591683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.374862+0000","last_change":"2023-09-30T00:55:55.373189+0000","last_active":"2023-09-30T00:55:55.374862+0000","last_peered":"2023-09-30T00:55:55.374862+0000","last_clean":"2023-09-30T00:55:55.374862+0000","last_became_active":"2023-09-30T00:55:55.372455+0000","last_became_peered":"2023-09-30T00:55:55.372455+0000","last_unstale":"2023-09-30T00:55:55.374862+0000","last_undegraded":"2023-09-30T00:55:55.374862+0000","last_fullsized":"2023-09-30T00:55:55.374862+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:20.834983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.372137+0000","last_change":"2023-09-30T00:55:55.372137+0000","last_active":"2023-09-30T00:55:55.372137+0000","last_peered":"2023-09-30T00:55:55.372137+0000","last_clean":"2023-09-30T00:55:55.372137+0000","last_became_active":"2023-09-30T00:55:55.371497+0000","last_became_peered":"2023-09-30T00:55:55.371497+0000","last_unstale":"2023-09-30T00:55:55.372137+0000","last_undegraded":"2023-09-30T00:55:55.372137+0000","last_fullsized":"2023-09-30T00:55:55.372137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:11.573196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.352993+0000","last_change":"2023-09-30T00:55:55.351405+0000","last_active":"2023-09-30T00:55:55.352993+0000","last_peered":"2023-09-30T00:55:55.352993+0000","last_clean":"2023-09-30T00:55:55.352993+0000","last_became_active":"2023-09-30T00:55:55.350964+0000","last_became_peered":"2023-09-30T00:55:55.350964+0000","last_unstale":"2023-09-30T00:55:55.352993+0000","last_undegraded":"2023-09-30T00:55:55.352993+0000","last_fullsized":"2023-09-30T00:55:55.352993+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:22.552075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.342427+0000","last_change":"2023-09-30T00:55:55.342427+0000","last_active":"2023-09-30T00:55:55.342427+0000","last_peered":"2023-09-30T00:55:55.342427+0000","last_clean":"2023-09-30T00:55:55.342427+0000","last_became_active":"2023-09-30T00:55:55.342131+0000","last_became_peered":"2023-09-30T00:55:55.342131+0000","last_unstale":"2023-09-30T00:55:55.342427+0000","last_undegraded":"2023-09-30T00:55:55.342427+0000","last_fullsized":"2023-09-30T00:55:55.342427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:06.525330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.353197+0000","last_change":"2023-09-30T00:55:55.351630+0000","last_active":"2023-09-30T00:55:55.353197+0000","last_peered":"2023-09-30T00:55:55.353197+0000","last_clean":"2023-09-30T00:55:55.353197+0000","last_became_active":"2023-09-30T00:55:55.350971+0000","last_became_peered":"2023-09-30T00:55:55.350971+0000","last_unstale":"2023-09-30T00:55:55.353197+0000","last_undegraded":"2023-09-30T00:55:55.353197+0000","last_fullsized":"2023-09-30T00:55:55.353197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:07.184194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.350955+0000","last_change":"2023-09-30T00:55:55.350955+0000","last_active":"2023-09-30T00:55:55.350955+0000","last_peered":"2023-09-30T00:55:55.350955+0000","last_clean":"2023-09-30T00:55:55.350955+0000","last_became_active":"2023-09-30T00:55:55.350574+0000","last_became_peered":"2023-09-30T00:55:55.350574+0000","last_unstale":"2023-09-30T00:55:55.350955+0000","last_undegraded":"2023-09-30T00:55:55.350955+0000","last_fullsized":"2023-09-30T00:55:55.350955+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:00.845664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.351517+0000","last_change":"2023-09-30T00:55:55.351517+0000","last_active":"2023-09-30T00:55:55.351517+0000","last_peered":"2023-09-30T00:55:55.351517+0000","last_clean":"2023-09-30T00:55:55.351517+0000","last_became_active":"2023-09-30T00:55:55.351023+0000","last_became_peered":"2023-09-30T00:55:55.351023+0000","last_unstale":"2023-09-30T00:55:55.351517+0000","last_undegraded":"2023-09-30T00:55:55.351517+0000","last_fullsized":"2023-09-30T00:55:55.351517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:11.323222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'105","reported_seq":146,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:56.058171+0000","last_change":"2023-09-30T00:55:54.335817+0000","last_active":"2023-09-30T00:55:56.058171+0000","last_peered":"2023-09-30T00:55:56.058171+0000","last_clean":"2023-09-30T00:55:56.058171+0000","last_became_active":"2023-09-30T00:55:54.335401+0000","last_became_peered":"2023-09-30T00:55:54.335401+0000","last_unstale":"2023-09-30T00:55:56.058171+0000","last_undegraded":"2023-09-30T00:55:56.058171+0000","last_fullsized":"2023-09-30T00:55:56.058171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:53.298069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:53.298069+0000","last_clean_scrub_stamp":"2023-09-30T00:55:53.298069+0000","objects_scrubbed":0,"log_size":105,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:22.054834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":95,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":95,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":1402464,"data_compressed":80991,"data_compressed_allocated":110592,"data_compressed_original":1351680,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5488,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366352,"statfs":{"total":96636764160,"available":96631144448,"internally_reserved":0,"allocated":217088,"data_stored":457145,"data_compressed":21418,"data_compressed_allocated":28672,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.013999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":47.002000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.486999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.15}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8620000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3410000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5488,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366352,"statfs":{"total":96636764160,"available":96631144448,"internally_reserved":0,"allocated":217088,"data_stored":457145,"data_compressed":21418,"data_compressed_allocated":28672,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5164,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366676,"statfs":{"total":96636764160,"available":96631476224,"internally_reserved":0,"allocated":167936,"data_stored":51629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5176,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366664,"statfs":{"total":96636764160,"available":96631463936,"internally_reserved":0,"allocated":180224,"data_stored":57365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.353}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":51629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5172,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366668,"statfs":{"total":96636764160,"available":96631468032,"internally_reserved":0,"allocated":176128,"data_stored":56409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5176,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366664,"statfs":{"total":96636764160,"available":96631463936,"internally_reserved":0,"allocated":180224,"data_stored":57365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5628,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366212,"statfs":{"total":96636764160,"available":96631001088,"internally_reserved":0,"allocated":225280,"data_stored":519117,"data_compressed":26997,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":467488,"data_compressed":26997,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":467488,"data_compressed":26997,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":467488,"data_compressed":26997,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T00:56:00.379 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T00:56:00.653 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:00.655 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-09-30T00:56:00.666 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2023-09-30T00:56:00.399549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":95,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":42392,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":40768,"kb_avail":754932328,"statfs":{"total":773094113280,"available":773050703872,"internally_reserved":0,"allocated":1531904,"data_stored":1707804,"data_compressed":69833,"data_compressed_allocated":94208,"data_compressed_original":1228800,"omap_allocated":0,"internal_metadata":41746432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.053096"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.344142+0000","last_change":"2023-09-30T00:55:55.344142+0000","last_active":"2023-09-30T00:55:55.344142+0000","last_peered":"2023-09-30T00:55:55.344142+0000","last_clean":"2023-09-30T00:55:55.344142+0000","last_became_active":"2023-09-30T00:55:55.343702+0000","last_became_peered":"2023-09-30T00:55:55.343702+0000","last_unstale":"2023-09-30T00:55:55.344142+0000","last_undegraded":"2023-09-30T00:55:55.344142+0000","last_fullsized":"2023-09-30T00:55:55.344142+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:10.591683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.374862+0000","last_change":"2023-09-30T00:55:55.373189+0000","last_active":"2023-09-30T00:55:55.374862+0000","last_peered":"2023-09-30T00:55:55.374862+0000","last_clean":"2023-09-30T00:55:55.374862+0000","last_became_active":"2023-09-30T00:55:55.372455+0000","last_became_peered":"2023-09-30T00:55:55.372455+0000","last_unstale":"2023-09-30T00:55:55.374862+0000","last_undegraded":"2023-09-30T00:55:55.374862+0000","last_fullsized":"2023-09-30T00:55:55.374862+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:20.834983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.372137+0000","last_change":"2023-09-30T00:55:55.372137+0000","last_active":"2023-09-30T00:55:55.372137+0000","last_peered":"2023-09-30T00:55:55.372137+0000","last_clean":"2023-09-30T00:55:55.372137+0000","last_became_active":"2023-09-30T00:55:55.371497+0000","last_became_peered":"2023-09-30T00:55:55.371497+0000","last_unstale":"2023-09-30T00:55:55.372137+0000","last_undegraded":"2023-09-30T00:55:55.372137+0000","last_fullsized":"2023-09-30T00:55:55.372137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:11.573196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.352993+0000","last_change":"2023-09-30T00:55:55.351405+0000","last_active":"2023-09-30T00:55:55.352993+0000","last_peered":"2023-09-30T00:55:55.352993+0000","last_clean":"2023-09-30T00:55:55.352993+0000","last_became_active":"2023-09-30T00:55:55.350964+0000","last_became_peered":"2023-09-30T00:55:55.350964+0000","last_unstale":"2023-09-30T00:55:55.352993+0000","last_undegraded":"2023-09-30T00:55:55.352993+0000","last_fullsized":"2023-09-30T00:55:55.352993+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:22.552075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.342427+0000","last_change":"2023-09-30T00:55:55.342427+0000","last_active":"2023-09-30T00:55:55.342427+0000","last_peered":"2023-09-30T00:55:55.342427+0000","last_clean":"2023-09-30T00:55:55.342427+0000","last_became_active":"2023-09-30T00:55:55.342131+0000","last_became_peered":"2023-09-30T00:55:55.342131+0000","last_unstale":"2023-09-30T00:55:55.342427+0000","last_undegraded":"2023-09-30T00:55:55.342427+0000","last_fullsized":"2023-09-30T00:55:55.342427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:06.525330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.353197+0000","last_change":"2023-09-30T00:55:55.351630+0000","last_active":"2023-09-30T00:55:55.353197+0000","last_peered":"2023-09-30T00:55:55.353197+0000","last_clean":"2023-09-30T00:55:55.353197+0000","last_became_active":"2023-09-30T00:55:55.350971+0000","last_became_peered":"2023-09-30T00:55:55.350971+0000","last_unstale":"2023-09-30T00:55:55.353197+0000","last_undegraded":"2023-09-30T00:55:55.353197+0000","last_fullsized":"2023-09-30T00:55:55.353197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:07.184194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.350955+0000","last_change":"2023-09-30T00:55:55.350955+0000","last_active":"2023-09-30T00:55:55.350955+0000","last_peered":"2023-09-30T00:55:55.350955+0000","last_clean":"2023-09-30T00:55:55.350955+0000","last_became_active":"2023-09-30T00:55:55.350574+0000","last_became_peered":"2023-09-30T00:55:55.350574+0000","last_unstale":"2023-09-30T00:55:55.350955+0000","last_undegraded":"2023-09-30T00:55:55.350955+0000","last_fullsized":"2023-09-30T00:55:55.350955+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:00.845664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:55.351517+0000","last_change":"2023-09-30T00:55:55.351517+0000","last_active":"2023-09-30T00:55:55.351517+0000","last_peered":"2023-09-30T00:55:55.351517+0000","last_clean":"2023-09-30T00:55:55.351517+0000","last_became_active":"2023-09-30T00:55:55.351023+0000","last_became_peered":"2023-09-30T00:55:55.351023+0000","last_unstale":"2023-09-30T00:55:55.351517+0000","last_undegraded":"2023-09-30T00:55:55.351517+0000","last_fullsized":"2023-09-30T00:55:55.351517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:55:54.309464+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:11.323222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'105","reported_seq":146,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T00:55:56.058171+0000","last_change":"2023-09-30T00:55:54.335817+0000","last_active":"2023-09-30T00:55:56.058171+0000","last_peered":"2023-09-30T00:55:56.058171+0000","last_clean":"2023-09-30T00:55:56.058171+0000","last_became_active":"2023-09-30T00:55:54.335401+0000","last_became_peered":"2023-09-30T00:55:54.335401+0000","last_unstale":"2023-09-30T00:55:56.058171+0000","last_undegraded":"2023-09-30T00:55:56.058171+0000","last_fullsized":"2023-09-30T00:55:56.058171+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:55:53.298069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:53.298069+0000","last_clean_scrub_stamp":"2023-09-30T00:55:53.298069+0000","objects_scrubbed":0,"log_size":105,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T00:57:22.054834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":95,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":95,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":1402464,"data_compressed":80991,"data_compressed_allocated":110592,"data_compressed_original":1351680,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5488,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366352,"statfs":{"total":96636764160,"available":96631144448,"internally_reserved":0,"allocated":217088,"data_stored":457145,"data_compressed":21418,"data_compressed_allocated":28672,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.013999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":47.002000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.486999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.15}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8620000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3410000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5488,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366352,"statfs":{"total":96636764160,"available":96631144448,"internally_reserved":0,"allocated":217088,"data_stored":457145,"data_compressed":21418,"data_compressed_allocated":28672,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5164,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366676,"statfs":{"total":96636764160,"available":96631476224,"internally_reserved":0,"allocated":167936,"data_stored":51629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5176,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366664,"statfs":{"total":96636764160,"available":96631463936,"internally_reserved":0,"allocated":180224,"data_stored":57365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.353}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":51629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5172,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366668,"statfs":{"total":96636764160,"available":96631468032,"internally_reserved":0,"allocated":176128,"data_stored":56409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5176,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366664,"statfs":{"total":96636764160,"available":96631463936,"internally_reserved":0,"allocated":180224,"data_stored":57365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5628,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366212,"statfs":{"total":96636764160,"available":96631001088,"internally_reserved":0,"allocated":225280,"data_stored":519117,"data_compressed":26997,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":467488,"data_compressed":26997,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":467488,"data_compressed":26997,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":467488,"data_compressed":26997,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T00:56:00.668 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-30T00:56:00.668 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-30T00:56:00.668 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-30T00:56:00.668 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:00.980 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:00.980 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:04.000 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:04.336 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:04.336 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:07.351 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:07.688 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:07.688 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:10.703 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:11.042 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:11.042 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:14.062 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:14.391 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:14.391 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:17.406 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:17.741 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:17.741 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:20.755 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:21.099 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:21.100 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:24.114 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:24.444 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:24.444 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:27.461 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:27.797 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:27.797 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:30.811 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:31.143 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:31.144 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:34.159 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:34.496 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:34.496 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:37.511 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:37.846 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:37.847 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:40.862 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:41.197 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:44.211 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:44.551 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:44.552 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:47.567 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:47.902 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:47.902 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:50.922 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:51.246 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:51.246 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:54.261 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:54.595 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:54.595 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:56:57.610 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:56:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:56:57.950 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:00.964 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:01.300 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:01.300 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:04.317 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:04.644 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:04.644 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:07.662 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:07.994 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:11.011 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:11.339 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:11.340 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:14.357 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:14.681 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:14.681 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:17.696 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:18.029 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:18.030 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:21.043 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:21.381 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:21.381 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:24.395 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:24.729 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:24.729 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:27.743 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:28.065 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:28.066 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:31.084 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:31.423 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:31.424 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:34.440 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:34.768 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:34.768 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:37.783 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:38.111 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:38.111 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:41.127 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:41.458 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:44.473 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:44.805 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:44.805 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:47.819 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:48.151 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:48.151 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:51.170 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:51.499 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:51.499 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:54.514 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:54.849 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:54.850 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:57:57.864 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:57:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:57:58.199 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:01.218 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:01.546 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:01.547 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:04.561 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:04.887 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:04.887 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:07.901 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:08.227 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:08.227 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:11.242 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:11.572 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:11.572 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:14.594 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:14.926 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:14.927 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:17.941 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:18.275 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:18.276 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:21.290 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:21.626 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:21.626 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:24.640 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:24.979 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:24.979 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:27.998 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:28.328 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:28.328 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:31.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:31.683 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:31.683 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:34.699 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:35.027 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:35.027 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:38.047 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:38.377 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:38.377 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:41.392 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:41.723 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:44.737 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:45.067 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:45.067 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:48.081 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:48.415 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:48.415 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:51.429 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:51.760 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:51.761 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:54.775 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:55.112 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:55.112 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:58:58.133 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:58:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:58:58.468 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:01.480 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:01.812 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:01.812 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:04.827 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:05.160 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:05.160 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:08.175 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:08.501 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:08.501 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:11.515 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:11.844 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:11.844 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:14.859 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:15.198 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:15.198 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:18.213 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:18.539 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:18.539 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:21.553 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:21.884 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:21.884 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:24.899 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:25.238 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:25.238 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:28.252 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:28.575 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:28.575 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:31.586 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:31.914 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:31.915 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:34.930 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:35.261 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:35.262 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:38.276 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:38.611 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:38.611 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:41.626 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:41.955 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:44.970 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:45.306 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:45.307 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:48.321 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:48.652 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:48.653 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:51.667 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:52.005 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:52.005 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:55.024 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:55.354 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:55.354 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T00:59:58.372 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T00:59:58.704 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T00:59:58.704 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:00.000 INFO:tasks.ceph.mon.a.smithi069.stderr:2023-09-30T00:59:59.997+0000 7fa1fe3da700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-30T01:00:01.721 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:02.052 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:02.053 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:05.068 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:05.394 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:05.394 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:08.409 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:08.743 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:08.743 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:11.757 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:12.095 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:12.095 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:15.109 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:15.443 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:15.443 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:18.457 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:18.787 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:21.802 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:22.150 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:22.150 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:25.166 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:25.498 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:25.499 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:28.517 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:28.850 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:28.850 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:31.871 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:32.204 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:32.204 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:35.220 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:35.551 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:35.551 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:38.565 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:38.905 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:38.905 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:41.919 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:42.266 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:42.270 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:45.281 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:45.608 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:45.608 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:48.622 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:48.958 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:48.958 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:51.978 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:52.305 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:52.306 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:55.320 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:55.643 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:00:58.658 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:00:58.990 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:00:58.990 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:01:02.003 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T01:01:02.340 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:01:02.340 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-30T01:01:02.355 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-09-30T01:01:02.356 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T01:01:02.652 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:01:02.652 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":43,"fsid":"a4f1df1b-c238-45bb-b595-c54d4bb08739","created":"2023-09-30T00:55:42.211215+0000","modified":"2023-09-30T01:00:50.289802+0000","last_up_change":"2023-09-30T00:55:51.275589+0000","last_in_change":"2023-09-30T00:55:44.646942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T00:55:52.460922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T00:55:53.353727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3644de55-4abe-484a-ab81-37b6248b978c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":31243},{"type":"v1","addr":"172.21.15.69:6817","nonce":31243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":31243},{"type":"v1","addr":"172.21.15.69:6819","nonce":31243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":31243},{"type":"v1","addr":"172.21.15.69:6823","nonce":31243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":31243},{"type":"v1","addr":"172.21.15.69:6821","nonce":31243}]},"public_addr":"172.21.15.69:6817/31243","cluster_addr":"172.21.15.69:6819/31243","heartbeat_back_addr":"172.21.15.69:6823/31243","heartbeat_front_addr":"172.21.15.69:6821/31243","state":["exists","up"]},{"osd":1,"uuid":"e81ee49e-57c0-4bd4-81e1-943a53e17e0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":31242},{"type":"v1","addr":"172.21.15.69:6825","nonce":31242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":31242},{"type":"v1","addr":"172.21.15.69:6827","nonce":31242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":31242},{"type":"v1","addr":"172.21.15.69:6831","nonce":31242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":31242},{"type":"v1","addr":"172.21.15.69:6829","nonce":31242}]},"public_addr":"172.21.15.69:6825/31242","cluster_addr":"172.21.15.69:6827/31242","heartbeat_back_addr":"172.21.15.69:6831/31242","heartbeat_front_addr":"172.21.15.69:6829/31242","state":["exists","up"]},{"osd":2,"uuid":"95f49092-0cb9-4fcf-8ec8-740061c438c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":31245},{"type":"v1","addr":"172.21.15.69:6809","nonce":31245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":31245},{"type":"v1","addr":"172.21.15.69:6811","nonce":31245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":31245},{"type":"v1","addr":"172.21.15.69:6815","nonce":31245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":31245},{"type":"v1","addr":"172.21.15.69:6813","nonce":31245}]},"public_addr":"172.21.15.69:6809/31245","cluster_addr":"172.21.15.69:6811/31245","heartbeat_back_addr":"172.21.15.69:6815/31245","heartbeat_front_addr":"172.21.15.69:6813/31245","state":["exists","up"]},{"osd":3,"uuid":"22d0daa1-283e-477d-8088-31922a3ff54f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":31241},{"type":"v1","addr":"172.21.15.69:6801","nonce":31241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":31241},{"type":"v1","addr":"172.21.15.69:6803","nonce":31241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":31241},{"type":"v1","addr":"172.21.15.69:6807","nonce":31241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":31241},{"type":"v1","addr":"172.21.15.69:6805","nonce":31241}]},"public_addr":"172.21.15.69:6801/31241","cluster_addr":"172.21.15.69:6803/31241","heartbeat_back_addr":"172.21.15.69:6807/31241","heartbeat_front_addr":"172.21.15.69:6805/31241","state":["exists","up"]},{"osd":4,"uuid":"601a722e-5cd4-46b0-9651-74d99ab73be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":31269},{"type":"v1","addr":"172.21.15.150:6801","nonce":31269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":31269},{"type":"v1","addr":"172.21.15.150:6803","nonce":31269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":31269},{"type":"v1","addr":"172.21.15.150:6807","nonce":31269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":31269},{"type":"v1","addr":"172.21.15.150:6805","nonce":31269}]},"public_addr":"172.21.15.150:6801/31269","cluster_addr":"172.21.15.150:6803/31269","heartbeat_back_addr":"172.21.15.150:6807/31269","heartbeat_front_addr":"172.21.15.150:6805/31269","state":["exists","up"]},{"osd":5,"uuid":"c726bca0-dcd7-4041-88b3-6234ddb84477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":31270},{"type":"v1","addr":"172.21.15.150:6809","nonce":31270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":31270},{"type":"v1","addr":"172.21.15.150:6811","nonce":31270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":31270},{"type":"v1","addr":"172.21.15.150:6815","nonce":31270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":31270},{"type":"v1","addr":"172.21.15.150:6813","nonce":31270}]},"public_addr":"172.21.15.150:6809/31270","cluster_addr":"172.21.15.150:6811/31270","heartbeat_back_addr":"172.21.15.150:6815/31270","heartbeat_front_addr":"172.21.15.150:6813/31270","state":["exists","up"]},{"osd":6,"uuid":"9dc1bc9b-177d-448c-8917-2c400d1d3ca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":31271},{"type":"v1","addr":"172.21.15.150:6825","nonce":31271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":31271},{"type":"v1","addr":"172.21.15.150:6827","nonce":31271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":31271},{"type":"v1","addr":"172.21.15.150:6831","nonce":31271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":31271},{"type":"v1","addr":"172.21.15.150:6829","nonce":31271}]},"public_addr":"172.21.15.150:6825/31271","cluster_addr":"172.21.15.150:6827/31271","heartbeat_back_addr":"172.21.15.150:6831/31271","heartbeat_front_addr":"172.21.15.150:6829/31271","state":["exists","up"]},{"osd":7,"uuid":"45e5f524-aa7c-46a7-80a1-bfbeac544530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":31268},{"type":"v1","addr":"172.21.15.150:6817","nonce":31268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":31268},{"type":"v1","addr":"172.21.15.150:6819","nonce":31268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":31268},{"type":"v1","addr":"172.21.15.150:6823","nonce":31268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":31268},{"type":"v1","addr":"172.21.15.150:6821","nonce":31268}]},"public_addr":"172.21.15.150:6817/31268","cluster_addr":"172.21.15.150:6819/31268","heartbeat_back_addr":"172.21.15.150:6823/31268","heartbeat_front_addr":"172.21.15.150:6821/31268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:32.472034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:39.260646+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:22.573301+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:21.541272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:59:48.738371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:59:58.447620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:45.169826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:49.927449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T01:01:02.667 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T01:01:02.667 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:01:02.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:01:02.966 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-09-30T01:01:02.978 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2023-09-30T01:01:02.489631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":47184,"kb_used_data":3984,"kb_used_omap":0,"kb_used_meta":43136,"kb_avail":754927536,"statfs":{"total":773094113280,"available":773045796864,"internally_reserved":0,"allocated":4079616,"data_stored":3136348,"data_compressed":89871,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":44171264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003428"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.280356+0000","last_change":"2023-09-30T01:00:02.106924+0000","last_active":"2023-09-30T01:00:45.280356+0000","last_peered":"2023-09-30T01:00:45.280356+0000","last_clean":"2023-09-30T01:00:45.280356+0000","last_became_active":"2023-09-30T00:55:55.343702+0000","last_became_peered":"2023-09-30T00:55:55.343702+0000","last_unstale":"2023-09-30T01:00:45.280356+0000","last_undegraded":"2023-09-30T01:00:45.280356+0000","last_fullsized":"2023-09-30T01:00:45.280356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:02.106695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:11.864778+0000","last_clean_scrub_stamp":"2023-09-30T01:00:02.106695+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:11.197424+0000","scrub_duration":0.0047035439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.992512+0000","last_change":"2023-09-30T01:00:00.083018+0000","last_active":"2023-09-30T01:00:45.992512+0000","last_peered":"2023-09-30T01:00:45.992512+0000","last_clean":"2023-09-30T01:00:45.992512+0000","last_became_active":"2023-09-30T00:55:55.372455+0000","last_became_peered":"2023-09-30T00:55:55.372455+0000","last_unstale":"2023-09-30T01:00:45.992512+0000","last_undegraded":"2023-09-30T01:00:45.992512+0000","last_fullsized":"2023-09-30T01:00:45.992512+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:00.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T01:00:00.082925+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:05.167676+0000","scrub_duration":0.0039526120000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":64,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993907+0000","last_change":"2023-09-30T00:59:52.818842+0000","last_active":"2023-09-30T01:00:45.993907+0000","last_peered":"2023-09-30T01:00:45.993907+0000","last_clean":"2023-09-30T01:00:45.993907+0000","last_became_active":"2023-09-30T00:55:55.371497+0000","last_became_peered":"2023-09-30T00:55:55.371497+0000","last_unstale":"2023-09-30T01:00:45.993907+0000","last_undegraded":"2023-09-30T01:00:45.993907+0000","last_fullsized":"2023-09-30T01:00:45.993907+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:52.818607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:59:52.818607+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:15.647590+0000","scrub_duration":0.0043827509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.992377+0000","last_change":"2023-09-30T01:00:11.015444+0000","last_active":"2023-09-30T01:00:45.992377+0000","last_peered":"2023-09-30T01:00:45.992377+0000","last_clean":"2023-09-30T01:00:45.992377+0000","last_became_active":"2023-09-30T00:55:55.350964+0000","last_became_peered":"2023-09-30T00:55:55.350964+0000","last_unstale":"2023-09-30T01:00:45.992377+0000","last_undegraded":"2023-09-30T01:00:45.992377+0000","last_fullsized":"2023-09-30T01:00:45.992377+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:11.015207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T01:00:11.015207+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:38.038876+0000","scrub_duration":0.0042614180000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993012+0000","last_change":"2023-09-30T00:59:56.922793+0000","last_active":"2023-09-30T01:00:45.993012+0000","last_peered":"2023-09-30T01:00:45.993012+0000","last_clean":"2023-09-30T01:00:45.993012+0000","last_became_active":"2023-09-30T00:55:55.342131+0000","last_became_peered":"2023-09-30T00:55:55.342131+0000","last_unstale":"2023-09-30T01:00:45.993012+0000","last_undegraded":"2023-09-30T01:00:45.993012+0000","last_fullsized":"2023-09-30T01:00:45.993012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:56.922501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:06.562848+0000","last_clean_scrub_stamp":"2023-09-30T00:59:56.922501+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:24.747687+0000","scrub_duration":0.0040857139999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.277663+0000","last_change":"2023-09-30T00:59:48.922555+0000","last_active":"2023-09-30T01:00:45.277663+0000","last_peered":"2023-09-30T01:00:45.277663+0000","last_clean":"2023-09-30T01:00:45.277663+0000","last_became_active":"2023-09-30T00:55:55.350971+0000","last_became_peered":"2023-09-30T00:55:55.350971+0000","last_unstale":"2023-09-30T01:00:45.277663+0000","last_undegraded":"2023-09-30T01:00:45.277663+0000","last_fullsized":"2023-09-30T01:00:45.277663+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:48.922299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:58:20.471763+0000","last_clean_scrub_stamp":"2023-09-30T00:59:48.922299+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:11.800573+0000","scrub_duration":0.0043283000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":64,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993821+0000","last_change":"2023-09-30T00:59:49.817759+0000","last_active":"2023-09-30T01:00:45.993821+0000","last_peered":"2023-09-30T01:00:45.993821+0000","last_clean":"2023-09-30T01:00:45.993821+0000","last_became_active":"2023-09-30T00:55:55.350574+0000","last_became_peered":"2023-09-30T00:55:55.350574+0000","last_unstale":"2023-09-30T01:00:45.993821+0000","last_undegraded":"2023-09-30T01:00:45.993821+0000","last_fullsized":"2023-09-30T01:00:45.993821+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:49.817496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:59:49.817496+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:11.184156+0000","scrub_duration":0.0079735229999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993084+0000","last_change":"2023-09-30T00:59:37.953173+0000","last_active":"2023-09-30T01:00:45.993084+0000","last_peered":"2023-09-30T01:00:45.993084+0000","last_clean":"2023-09-30T01:00:45.993084+0000","last_became_active":"2023-09-30T00:55:55.351023+0000","last_became_peered":"2023-09-30T00:55:55.351023+0000","last_unstale":"2023-09-30T01:00:45.993084+0000","last_undegraded":"2023-09-30T01:00:45.993084+0000","last_fullsized":"2023-09-30T01:00:45.993084+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:37.952950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:13.467244+0000","last_clean_scrub_stamp":"2023-09-30T00:59:37.952950+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:07.858959+0000","scrub_duration":0.0040933410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":216,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993903+0000","last_change":"2023-09-30T01:00:04.742088+0000","last_active":"2023-09-30T01:00:45.993903+0000","last_peered":"2023-09-30T01:00:45.993903+0000","last_clean":"2023-09-30T01:00:45.993903+0000","last_became_active":"2023-09-30T00:55:54.335401+0000","last_became_peered":"2023-09-30T00:55:54.335401+0000","last_unstale":"2023-09-30T01:00:45.993903+0000","last_undegraded":"2023-09-30T01:00:45.993903+0000","last_fullsized":"2023-09-30T01:00:45.993903+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'117","last_scrub_stamp":"2023-09-30T01:00:04.741859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:53.298069+0000","last_clean_scrub_stamp":"2023-09-30T01:00:04.741859+0000","objects_scrubbed":2,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:16.702808+0000","scrub_duration":0.0048182659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":1586784,"data_compressed":89871,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6240,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365600,"statfs":{"total":96636764160,"available":96630374400,"internally_reserved":0,"allocated":548864,"data_stored":722743,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":1.054,"15min":1.514},"min":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.135,"5min":18.013999999999999,"15min":18.013999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.127,"15min":1.6519999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.145,"5min":20.856999999999999,"15min":20.856999999999999},"last":0.873}]},{"osd":1,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.849,"15min":4.306},"min":{"1min":0.63800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.24,"5min":50.606999999999999,"15min":50.606999999999999},"last":1.01},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.357,"15min":2.4209999999999998},"min":{"1min":0.70699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4199999999999999,"5min":41.756999999999998,"15min":41.756999999999998},"last":0.72899999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.1739999999999999,"15min":1.641},"min":{"1min":0.83699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.1080000000000001,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":23.733000000000001,"5min":8.5530000000000008,"15min":4.1509999999999998},"min":{"1min":0.86899999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":364.00400000000002,"5min":364.00400000000002,"15min":364.00400000000002},"last":0.94899999999999995}]},{"osd":3,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":39.890000000000001,"5min":9.0020000000000007,"15min":4.1059999999999999},"min":{"1min":0.626,"5min":0.621,"15min":0.621},"max":{"1min":501.76100000000002,"5min":501.76100000000002,"15min":501.76100000000002},"last":1.101},{"interface":"front","average":{"1min":30.462,"5min":7.0800000000000001,"15min":3.5630000000000002},"min":{"1min":0.80800000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":501.42500000000001,"5min":501.42500000000001,"15min":501.42500000000001},"last":0.77100000000000002}]},{"osd":4,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.03,"15min":1.1819999999999999},"min":{"1min":0.70399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.482,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":15.436,"15min":6.0030000000000001},"min":{"1min":0.748,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2170000000000001,"5min":829.03899999999999,"15min":829.03899999999999},"last":1.028}]},{"osd":5,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":8.3789999999999996,"15min":3.7149999999999999},"min":{"1min":0.84899999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.393,"5min":830.25300000000004,"15min":830.25300000000004},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":6.4749999999999996,"15min":3.1070000000000002},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.2330000000000001,"5min":530.12199999999996,"15min":530.12199999999996},"last":1.133}]},{"osd":6,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":9.6679999999999993,"15min":4.21},"min":{"1min":0.86699999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.3080000000000001,"5min":971.51400000000001,"15min":971.51400000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.2110000000000001,"15min":1.359},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.2370000000000001,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":0.98399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6240,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365600,"statfs":{"total":96636764160,"available":96630374400,"internally_reserved":0,"allocated":548864,"data_stored":722743,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":32.222999999999999,"5min":7.2439999999999998,"15min":3.0750000000000002},"min":{"1min":0.71999999999999997,"5min":0.627,"15min":0.627},"max":{"1min":500.60300000000001,"5min":500.60300000000001,"15min":500.60300000000001},"last":1.123},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.94699999999999995,"15min":0.94199999999999995},"min":{"1min":0.69699999999999995,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4470000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.095}]},{"osd":1,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":10.343999999999999,"15min":4.0540000000000003},"min":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.369,"5min":688.31500000000005,"15min":688.31500000000005},"last":1.02},{"interface":"front","average":{"1min":0.95799999999999996,"5min":5.7679999999999998,"15min":2.6120000000000001},"min":{"1min":0.69099999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.248,"5min":501.56200000000001,"15min":501.56200000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":2.9100000000000001,"15min":1.6399999999999999},"min":{"1min":0.72499999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3149999999999999,"5min":201.053,"15min":201.053},"last":1.204},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0049999999999999,"15min":0.98799999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.335,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.94999999999999996}]},{"osd":3,"last update":"Sat Sep 30 00:59:59 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":8.2810000000000006,"15min":3.4790000000000001},"min":{"1min":0.82799999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.3939999999999999,"5min":756.92499999999995,"15min":756.92499999999995},"last":1.161},{"interface":"front","average":{"1min":54.628,"5min":21.196000000000002,"15min":7.766},"min":{"1min":0.80200000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":859.12800000000004,"5min":859.12800000000004,"15min":859.12800000000004},"last":0.98099999999999998}]},{"osd":4,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":7.3449999999999998,"15min":3.117},"min":{"1min":0.754,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4139999999999999,"5min":358.13099999999997,"15min":358.13099999999997},"last":1.327},{"interface":"front","average":{"1min":1.0569999999999999,"5min":6.0759999999999996,"15min":2.7120000000000002},"min":{"1min":0.80100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.246,"5min":528.37900000000002,"15min":528.37900000000002},"last":1.2330000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":39.414000000000001,"5min":8.7230000000000008,"15min":3.589},"min":{"1min":0.96699999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":651.43100000000004,"5min":651.43100000000004,"15min":651.43100000000004},"last":1.363},{"interface":"front","average":{"1min":1.069,"5min":1.0900000000000001,"15min":1.087},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.3220000000000001,"5min":1.641,"15min":1.641},"last":0.91600000000000004}]},{"osd":7,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1759999999999999,"15min":1.1799999999999999},"min":{"1min":0.84999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.492,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.1850000000000001,"15min":1.181},"min":{"1min":1.0169999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.3049999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.069}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5668,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366172,"statfs":{"total":96636764160,"available":96630960128,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.018,"15min":1.052},"min":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2190000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.022,"15min":1.0409999999999999},"min":{"1min":0.79400000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2529999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.135}]},{"osd":1,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.94099999999999995,"15min":0.92900000000000005},"min":{"1min":0.71999999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.2789999999999999,"5min":1.331,"15min":1.331},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":17.199000000000002,"15min":6.3840000000000003},"min":{"1min":0.66600000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1200000000000001,"5min":897.03999999999996,"15min":897.03999999999996},"last":1.1120000000000001}]},{"osd":2,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":18.686,"15min":36.369999999999997},"min":{"1min":0.746,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.1859999999999999,"5min":883.96900000000005,"15min":883.96900000000005},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.097,"15min":1.1180000000000001},"min":{"1min":0.81999999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.2609999999999999,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.96999999999999997}]},{"osd":3,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.069,"15min":1.085},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.232,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":6.298,"5min":2.141,"15min":1.45},"min":{"1min":0.80100000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":100.221,"5min":100.221,"15min":100.221},"last":1.0780000000000001}]},{"osd":4,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":3.6760000000000002,"15min":1.8700000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2869999999999999,"5min":272.95100000000002,"15min":272.95100000000002},"last":0.755},{"interface":"front","average":{"1min":1.081,"5min":1.1060000000000001,"15min":1.1539999999999999},"min":{"1min":0.79300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3360000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.94899999999999995}]},{"osd":6,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":4.0129999999999999,"15min":2.0590000000000002},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.28,"5min":309.56200000000001,"15min":309.56200000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2450000000000001,"5min":3.254,"15min":1.8680000000000001},"min":{"1min":0.90800000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.415,"5min":216.988,"15min":216.988},"last":0.88700000000000001}]},{"osd":7,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":7.7210000000000001,"15min":3.3849999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3680000000000001,"5min":609.06899999999996,"15min":609.06899999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":10.128,"15min":19.038},"min":{"1min":1.012,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.496,"5min":446.64499999999998,"15min":446.64499999999998},"last":1.0469999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5668,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366172,"statfs":{"total":96636764160,"available":96630960128,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":10.25,"15min":19.495999999999999},"min":{"1min":0.753,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.3,"5min":418.61799999999999,"15min":418.61799999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.085,"15min":1.0529999999999999},"min":{"1min":0.82799999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.411,"5min":1.48,"15min":1.48},"last":1.0109999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.016,"15min":1.022},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.296,"5min":1.421,"15min":1.421},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.153,"5min":1.155,"15min":1.1499999999999999},"min":{"1min":0.872,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.321,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1180000000000001}]},{"osd":2,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.98099999999999998,"15min":0.93600000000000005},"min":{"1min":0.754,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.071},{"interface":"front","average":{"1min":1.1459999999999999,"5min":12.583,"15min":10.872},"min":{"1min":0.88600000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.665,"5min":633.17700000000002,"15min":633.17700000000002},"last":1.161}]},{"osd":3,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":27.902000000000001,"15min":43.999000000000002},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.244,"5min":918.53399999999999,"15min":918.53399999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0169999999999999,"15min":0.999},"min":{"1min":0.76000000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4139999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0489999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.1970000000000001,"15min":1.139},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":16.021999999999998,"5min":16.021999999999998,"15min":16.021999999999998},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.018,"15min":1.03},"min":{"1min":0.71099999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3029999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.94299999999999995}]},{"osd":6,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":9.1929999999999996,"15min":14.262},"min":{"1min":0.77300000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.478,"5min":299.57100000000003,"15min":299.57100000000003},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.175,"5min":1.2050000000000001,"15min":1.2090000000000001},"min":{"1min":0.76800000000000002,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.369,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.2430000000000001}]},{"osd":7,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":4.8760000000000003,"5min":1.8939999999999999,"15min":1.415},"min":{"1min":0.78900000000000003,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":88.852000000000004,"5min":88.852000000000004,"15min":88.852000000000004},"last":1.097},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.181,"15min":1.194},"min":{"1min":0.82199999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.6259999999999999,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.82799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5668,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366172,"statfs":{"total":96636764160,"available":96630960128,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.71699999999999997},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76000000000000001,"15min":0.748},"min":{"1min":0.52300000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0249999999999999,"5min":1.073,"15min":1.073},"last":0.78200000000000003}]},{"osd":1,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":49.259,"5min":10.503,"15min":4.0309999999999997},"min":{"1min":0.64700000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":873.54200000000003,"5min":873.54200000000003,"15min":873.54200000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.85699999999999998,"15min":0.86499999999999999},"min":{"1min":0.59099999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.98499999999999999,"5min":1.607,"15min":1.607},"last":1.26}]},{"osd":2,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.89100000000000001,"15min":0.89600000000000002},"min":{"1min":0.66800000000000004,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1819999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91300000000000003,"15min":0.90300000000000002},"min":{"1min":0.72499999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.3220000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.05}]},{"osd":4,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":15.112,"15min":5.7560000000000002},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.115,"5min":735.82799999999997,"15min":735.82799999999997},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0960000000000001,"15min":1.1459999999999999},"min":{"1min":0.84999999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.24,"5min":1.875,"15min":1.875},"last":0.93700000000000006}]},{"osd":5,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":4.9619999999999997,"15min":2.4500000000000002},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.3480000000000001,"5min":363.78300000000002,"15min":363.78300000000002},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.1339999999999999,"15min":1.1639999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.226,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.1180000000000001}]},{"osd":6,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.1479999999999999,"15min":1.1819999999999999},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.321,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.038},{"interface":"front","average":{"1min":25.475999999999999,"5min":6.04,"15min":2.8239999999999998},"min":{"1min":0.94599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":462.90600000000001,"5min":462.90600000000001,"15min":462.90600000000001},"last":1.333}]},{"osd":7,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":18.411999999999999,"15min":6.9260000000000002},"min":{"1min":0.94299999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.3660000000000001,"5min":553.26900000000001,"15min":553.26900000000001},"last":1.161},{"interface":"front","average":{"1min":1.1379999999999999,"5min":19.024999999999999,"15min":27.608000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.399,"5min":583.47299999999996,"15min":583.47299999999996},"last":1.2290000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5728,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366112,"statfs":{"total":96636764160,"available":96630898688,"internally_reserved":0,"allocated":483328,"data_stored":192859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":3.1920000000000002,"15min":1.5609999999999999},"min":{"1min":0.53700000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.504,"5min":272.59399999999999,"15min":272.59399999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":6.5250000000000004,"15min":12.198},"min":{"1min":0.60199999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.206,"5min":298.29399999999998,"15min":298.29399999999998},"last":0.76400000000000001}]},{"osd":1,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":2.746,"15min":1.4379999999999999},"min":{"1min":0.54600000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.294,"5min":217.208,"15min":217.208},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":11.891,"15min":4.5350000000000001},"min":{"1min":0.68999999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2989999999999999,"5min":771.41800000000001,"15min":771.41800000000001},"last":0.85999999999999999}]},{"osd":3,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89100000000000001,"15min":0.93200000000000005},"min":{"1min":0.69099999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.028,"5min":1.782,"15min":1.782},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":5.6840000000000002,"15min":2.5640000000000001},"min":{"1min":0.72899999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8759999999999999,"5min":519.80100000000004,"15min":519.80100000000004},"last":0.99299999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.0609999999999999,"15min":1.087},"min":{"1min":0.78600000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":34.162999999999997,"5min":8.1120000000000001,"15min":3.4460000000000002},"min":{"1min":0.80100000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":501.62200000000001,"5min":501.62200000000001,"15min":501.62200000000001},"last":1.0409999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0569999999999999,"15min":1.079},"min":{"1min":0.752,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.3169999999999999,"5min":1.895,"15min":1.895},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.123,"5min":4.1070000000000002,"15min":2.1309999999999998},"min":{"1min":0.89400000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.9930000000000001,"5min":359.505,"15min":359.505},"last":0.94299999999999995}]},{"osd":6,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":8.6430000000000007,"15min":3.6850000000000001},"min":{"1min":0.84699999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.472,"5min":786.60400000000004,"15min":786.60400000000004},"last":1.069},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.84199999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.9410000000000001,"5min":1.944,"15min":1.944},"last":1.131}]},{"osd":7,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.238},"min":{"1min":0.88,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0310000000000001,"5min":2.048,"15min":2.048},"last":1.0920000000000001},{"interface":"front","average":{"1min":7.0119999999999996,"5min":2.3420000000000001,"15min":1.5940000000000001},"min":{"1min":0.98299999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":111.39400000000001,"5min":111.39400000000001,"15min":111.39400000000001},"last":1.163}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5732,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366108,"statfs":{"total":96636764160,"available":96630894592,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":9.9900000000000002,"15min":16.190000000000001},"min":{"1min":0.53700000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.321,"5min":553.65599999999995,"15min":553.65599999999995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":6.5229999999999997,"15min":2.7650000000000001},"min":{"1min":0.65800000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.087,"5min":501.34699999999998,"15min":501.34699999999998},"last":0.73099999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":4.0469999999999997,"15min":1.865},"min":{"1min":0.57499999999999996,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91300000000000003,"5min":313.49000000000001,"15min":313.49000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.86099999999999999,"15min":0.84299999999999997},"min":{"1min":0.60499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.282,"5min":1.319,"15min":1.319},"last":0.97199999999999998}]},{"osd":3,"last update":"Sat Sep 30 01:01:01 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.92400000000000004,"15min":0.91200000000000003},"min":{"1min":0.747,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.014},{"interface":"front","average":{"1min":55.460999999999999,"5min":31.306000000000001,"15min":11.065},"min":{"1min":0.78300000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":873.47900000000004,"5min":873.47900000000004,"15min":873.47900000000004},"last":873.47900000000004}]},{"osd":4,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.042,"15min":1.0880000000000001},"min":{"1min":0.76100000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.502,"5min":1.556,"15min":1.556},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1160000000000001,"15min":1.1180000000000001},"min":{"1min":0.88700000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.177}]},{"osd":5,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0529999999999999,"15min":1.089},"min":{"1min":0.83099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5169999999999999,"5min":1.71,"15min":1.71},"last":1.083},{"interface":"front","average":{"1min":37.758000000000003,"5min":10.032999999999999,"15min":6.3899999999999997},"min":{"1min":0.85499999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":586.91999999999996,"5min":586.91999999999996,"15min":586.91999999999996},"last":1.1319999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.123,"15min":1.161},"min":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.464,"5min":1.77,"15min":1.77},"last":1.246},{"interface":"front","average":{"1min":1.2110000000000001,"5min":6.8920000000000003,"15min":3.0840000000000001},"min":{"1min":1.0920000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.306,"5min":544.41600000000005,"15min":544.41600000000005},"last":1.28}]},{"osd":7,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.173,"15min":1.163},"min":{"1min":1.016,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.49,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.234,"5min":1.21,"15min":1.242},"min":{"1min":1.0549999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4279999999999999,"5min":1.861,"15min":1.861},"last":1.3360000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6240,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365600,"statfs":{"total":96636764160,"available":96630374400,"internally_reserved":0,"allocated":548864,"data_stored":722743,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":7.1059999999999999,"15min":9.6199999999999992},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.99299999999999999,"5min":309.20299999999997,"15min":309.20299999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":5.4189999999999996,"15min":2.3159999999999998},"min":{"1min":0.60799999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98099999999999998,"5min":533.17499999999995,"15min":533.17499999999995},"last":0.72599999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":14.385999999999999,"15min":27.942},"min":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0229999999999999,"5min":676.57399999999996,"15min":676.57399999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":24.475000000000001,"5min":18.829000000000001,"15min":6.859},"min":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":497.19900000000001,"5min":898.26700000000005,"15min":898.26700000000005},"last":0.78300000000000003}]},{"osd":3,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":4.0250000000000004,"15min":1.897},"min":{"1min":0.70599999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.214,"5min":362.38999999999999,"15min":362.38999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":1.0109999999999999},"min":{"1min":0.73499999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3360000000000001,"5min":1.585,"15min":1.585},"last":0.83599999999999997}]},{"osd":4,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.05,"15min":1.0569999999999999},"min":{"1min":0.90700000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.1919999999999999,"5min":1.377,"15min":1.377},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.117,"15min":1.1539999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.21,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.91200000000000003}]},{"osd":5,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":12.81,"15min":5.0179999999999998},"min":{"1min":0.79600000000000004,"5min":0.75,"15min":0.75},"max":{"1min":1.4059999999999999,"5min":667.67700000000002,"15min":667.67700000000002},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.107,"5min":4.2110000000000003,"15min":2.1120000000000001},"min":{"1min":0.97599999999999998,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.367,"5min":360.90100000000001,"15min":360.90100000000001},"last":1.232}]},{"osd":6,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":8.8379999999999992,"5min":2.6869999999999998,"15min":1.72},"min":{"1min":0.88900000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":163.465,"5min":163.465,"15min":163.465},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.05,"5min":14.26,"15min":5.5599999999999996},"min":{"1min":0.877,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.3540000000000001,"5min":970.15200000000004,"15min":970.15200000000004},"last":1.0409999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:00:04 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":7.8319999999999999,"15min":7.5570000000000004},"min":{"1min":0.88900000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.367,"5min":501.57299999999998,"15min":501.57299999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":22.666,"5min":13.086,"15min":5.1689999999999996},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":452.51799999999997,"5min":501.84800000000001,"15min":501.84800000000001},"last":1.2050000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:01:02.981 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:01:03.288 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-30T01:01:03.287+0000 7f8cfe376700 0 --2- 172.21.15.69:0/2521271153 >> [v2:172.21.15.69:3301/0,v1:172.21.15.69:6790/0] conn(0x7f8cf813dc20 0x7f8cf813e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-09-30T01:01:03.483 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:01:03.486 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-09-30T01:01:03.496 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2023-09-30T01:01:03.089552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":47184,"kb_used_data":3984,"kb_used_omap":0,"kb_used_meta":43136,"kb_avail":754927536,"statfs":{"total":773094113280,"available":773045796864,"internally_reserved":0,"allocated":4079616,"data_stored":3136348,"data_compressed":89871,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":44171264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.602802"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.280356+0000","last_change":"2023-09-30T01:00:02.106924+0000","last_active":"2023-09-30T01:00:45.280356+0000","last_peered":"2023-09-30T01:00:45.280356+0000","last_clean":"2023-09-30T01:00:45.280356+0000","last_became_active":"2023-09-30T00:55:55.343702+0000","last_became_peered":"2023-09-30T00:55:55.343702+0000","last_unstale":"2023-09-30T01:00:45.280356+0000","last_undegraded":"2023-09-30T01:00:45.280356+0000","last_fullsized":"2023-09-30T01:00:45.280356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:02.106695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:11.864778+0000","last_clean_scrub_stamp":"2023-09-30T01:00:02.106695+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:11.197424+0000","scrub_duration":0.0047035439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.992512+0000","last_change":"2023-09-30T01:00:00.083018+0000","last_active":"2023-09-30T01:00:45.992512+0000","last_peered":"2023-09-30T01:00:45.992512+0000","last_clean":"2023-09-30T01:00:45.992512+0000","last_became_active":"2023-09-30T00:55:55.372455+0000","last_became_peered":"2023-09-30T00:55:55.372455+0000","last_unstale":"2023-09-30T01:00:45.992512+0000","last_undegraded":"2023-09-30T01:00:45.992512+0000","last_fullsized":"2023-09-30T01:00:45.992512+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:00.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T01:00:00.082925+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:05.167676+0000","scrub_duration":0.0039526120000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":64,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993907+0000","last_change":"2023-09-30T00:59:52.818842+0000","last_active":"2023-09-30T01:00:45.993907+0000","last_peered":"2023-09-30T01:00:45.993907+0000","last_clean":"2023-09-30T01:00:45.993907+0000","last_became_active":"2023-09-30T00:55:55.371497+0000","last_became_peered":"2023-09-30T00:55:55.371497+0000","last_unstale":"2023-09-30T01:00:45.993907+0000","last_undegraded":"2023-09-30T01:00:45.993907+0000","last_fullsized":"2023-09-30T01:00:45.993907+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:52.818607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:59:52.818607+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:15.647590+0000","scrub_duration":0.0043827509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.992377+0000","last_change":"2023-09-30T01:00:11.015444+0000","last_active":"2023-09-30T01:00:45.992377+0000","last_peered":"2023-09-30T01:00:45.992377+0000","last_clean":"2023-09-30T01:00:45.992377+0000","last_became_active":"2023-09-30T00:55:55.350964+0000","last_became_peered":"2023-09-30T00:55:55.350964+0000","last_unstale":"2023-09-30T01:00:45.992377+0000","last_undegraded":"2023-09-30T01:00:45.992377+0000","last_fullsized":"2023-09-30T01:00:45.992377+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:11.015207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T01:00:11.015207+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:38.038876+0000","scrub_duration":0.0042614180000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993012+0000","last_change":"2023-09-30T00:59:56.922793+0000","last_active":"2023-09-30T01:00:45.993012+0000","last_peered":"2023-09-30T01:00:45.993012+0000","last_clean":"2023-09-30T01:00:45.993012+0000","last_became_active":"2023-09-30T00:55:55.342131+0000","last_became_peered":"2023-09-30T00:55:55.342131+0000","last_unstale":"2023-09-30T01:00:45.993012+0000","last_undegraded":"2023-09-30T01:00:45.993012+0000","last_fullsized":"2023-09-30T01:00:45.993012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:56.922501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:06.562848+0000","last_clean_scrub_stamp":"2023-09-30T00:59:56.922501+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:24.747687+0000","scrub_duration":0.0040857139999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.277663+0000","last_change":"2023-09-30T00:59:48.922555+0000","last_active":"2023-09-30T01:00:45.277663+0000","last_peered":"2023-09-30T01:00:45.277663+0000","last_clean":"2023-09-30T01:00:45.277663+0000","last_became_active":"2023-09-30T00:55:55.350971+0000","last_became_peered":"2023-09-30T00:55:55.350971+0000","last_unstale":"2023-09-30T01:00:45.277663+0000","last_undegraded":"2023-09-30T01:00:45.277663+0000","last_fullsized":"2023-09-30T01:00:45.277663+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:48.922299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:58:20.471763+0000","last_clean_scrub_stamp":"2023-09-30T00:59:48.922299+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:11.800573+0000","scrub_duration":0.0043283000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":64,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993821+0000","last_change":"2023-09-30T00:59:49.817759+0000","last_active":"2023-09-30T01:00:45.993821+0000","last_peered":"2023-09-30T01:00:45.993821+0000","last_clean":"2023-09-30T01:00:45.993821+0000","last_became_active":"2023-09-30T00:55:55.350574+0000","last_became_peered":"2023-09-30T00:55:55.350574+0000","last_unstale":"2023-09-30T01:00:45.993821+0000","last_undegraded":"2023-09-30T01:00:45.993821+0000","last_fullsized":"2023-09-30T01:00:45.993821+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:49.817496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:59:49.817496+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:11.184156+0000","scrub_duration":0.0079735229999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993084+0000","last_change":"2023-09-30T00:59:37.953173+0000","last_active":"2023-09-30T01:00:45.993084+0000","last_peered":"2023-09-30T01:00:45.993084+0000","last_clean":"2023-09-30T01:00:45.993084+0000","last_became_active":"2023-09-30T00:55:55.351023+0000","last_became_peered":"2023-09-30T00:55:55.351023+0000","last_unstale":"2023-09-30T01:00:45.993084+0000","last_undegraded":"2023-09-30T01:00:45.993084+0000","last_fullsized":"2023-09-30T01:00:45.993084+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:37.952950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:13.467244+0000","last_clean_scrub_stamp":"2023-09-30T00:59:37.952950+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:07.858959+0000","scrub_duration":0.0040933410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":216,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993903+0000","last_change":"2023-09-30T01:00:04.742088+0000","last_active":"2023-09-30T01:00:45.993903+0000","last_peered":"2023-09-30T01:00:45.993903+0000","last_clean":"2023-09-30T01:00:45.993903+0000","last_became_active":"2023-09-30T00:55:54.335401+0000","last_became_peered":"2023-09-30T00:55:54.335401+0000","last_unstale":"2023-09-30T01:00:45.993903+0000","last_undegraded":"2023-09-30T01:00:45.993903+0000","last_fullsized":"2023-09-30T01:00:45.993903+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'117","last_scrub_stamp":"2023-09-30T01:00:04.741859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:53.298069+0000","last_clean_scrub_stamp":"2023-09-30T01:00:04.741859+0000","objects_scrubbed":2,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:16.702808+0000","scrub_duration":0.0048182659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":1586784,"data_compressed":89871,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6240,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365600,"statfs":{"total":96636764160,"available":96630374400,"internally_reserved":0,"allocated":548864,"data_stored":722743,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":1.054,"15min":1.514},"min":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.135,"5min":18.013999999999999,"15min":18.013999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.127,"15min":1.6519999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.145,"5min":20.856999999999999,"15min":20.856999999999999},"last":0.873}]},{"osd":1,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.849,"15min":4.306},"min":{"1min":0.63800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.24,"5min":50.606999999999999,"15min":50.606999999999999},"last":1.01},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.357,"15min":2.4209999999999998},"min":{"1min":0.70699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4199999999999999,"5min":41.756999999999998,"15min":41.756999999999998},"last":0.72899999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.1739999999999999,"15min":1.641},"min":{"1min":0.83699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.1080000000000001,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":23.733000000000001,"5min":8.5530000000000008,"15min":4.1509999999999998},"min":{"1min":0.86899999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":364.00400000000002,"5min":364.00400000000002,"15min":364.00400000000002},"last":0.94899999999999995}]},{"osd":3,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":39.890000000000001,"5min":9.0020000000000007,"15min":4.1059999999999999},"min":{"1min":0.626,"5min":0.621,"15min":0.621},"max":{"1min":501.76100000000002,"5min":501.76100000000002,"15min":501.76100000000002},"last":1.101},{"interface":"front","average":{"1min":30.462,"5min":7.0800000000000001,"15min":3.5630000000000002},"min":{"1min":0.80800000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":501.42500000000001,"5min":501.42500000000001,"15min":501.42500000000001},"last":0.77100000000000002}]},{"osd":4,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.03,"15min":1.1819999999999999},"min":{"1min":0.70399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.482,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":15.436,"15min":6.0030000000000001},"min":{"1min":0.748,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2170000000000001,"5min":829.03899999999999,"15min":829.03899999999999},"last":1.028}]},{"osd":5,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":8.3789999999999996,"15min":3.7149999999999999},"min":{"1min":0.84899999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.393,"5min":830.25300000000004,"15min":830.25300000000004},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":6.4749999999999996,"15min":3.1070000000000002},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.2330000000000001,"5min":530.12199999999996,"15min":530.12199999999996},"last":1.133}]},{"osd":6,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":9.6679999999999993,"15min":4.21},"min":{"1min":0.86699999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.3080000000000001,"5min":971.51400000000001,"15min":971.51400000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.2110000000000001,"15min":1.359},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.2370000000000001,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":0.98399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6240,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365600,"statfs":{"total":96636764160,"available":96630374400,"internally_reserved":0,"allocated":548864,"data_stored":722743,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":32.222999999999999,"5min":7.2439999999999998,"15min":3.0750000000000002},"min":{"1min":0.71999999999999997,"5min":0.627,"15min":0.627},"max":{"1min":500.60300000000001,"5min":500.60300000000001,"15min":500.60300000000001},"last":1.123},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.94699999999999995,"15min":0.94199999999999995},"min":{"1min":0.69699999999999995,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4470000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.095}]},{"osd":1,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":10.343999999999999,"15min":4.0540000000000003},"min":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.369,"5min":688.31500000000005,"15min":688.31500000000005},"last":1.02},{"interface":"front","average":{"1min":0.95799999999999996,"5min":5.7679999999999998,"15min":2.6120000000000001},"min":{"1min":0.69099999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.248,"5min":501.56200000000001,"15min":501.56200000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":2.9100000000000001,"15min":1.6399999999999999},"min":{"1min":0.72499999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3149999999999999,"5min":201.053,"15min":201.053},"last":1.204},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0049999999999999,"15min":0.98799999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.335,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.94999999999999996}]},{"osd":3,"last update":"Sat Sep 30 00:59:59 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":8.2810000000000006,"15min":3.4790000000000001},"min":{"1min":0.82799999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.3939999999999999,"5min":756.92499999999995,"15min":756.92499999999995},"last":1.161},{"interface":"front","average":{"1min":54.628,"5min":21.196000000000002,"15min":7.766},"min":{"1min":0.80200000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":859.12800000000004,"5min":859.12800000000004,"15min":859.12800000000004},"last":0.98099999999999998}]},{"osd":4,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":7.3449999999999998,"15min":3.117},"min":{"1min":0.754,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4139999999999999,"5min":358.13099999999997,"15min":358.13099999999997},"last":1.327},{"interface":"front","average":{"1min":1.0569999999999999,"5min":6.0759999999999996,"15min":2.7120000000000002},"min":{"1min":0.80100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.246,"5min":528.37900000000002,"15min":528.37900000000002},"last":1.2330000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":39.414000000000001,"5min":8.7230000000000008,"15min":3.589},"min":{"1min":0.96699999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":651.43100000000004,"5min":651.43100000000004,"15min":651.43100000000004},"last":1.363},{"interface":"front","average":{"1min":1.069,"5min":1.0900000000000001,"15min":1.087},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.3220000000000001,"5min":1.641,"15min":1.641},"last":0.91600000000000004}]},{"osd":7,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1759999999999999,"15min":1.1799999999999999},"min":{"1min":0.84999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.492,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.1850000000000001,"15min":1.181},"min":{"1min":1.0169999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.3049999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.069}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5668,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366172,"statfs":{"total":96636764160,"available":96630960128,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.018,"15min":1.052},"min":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2190000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.022,"15min":1.0409999999999999},"min":{"1min":0.79400000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2529999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.135}]},{"osd":1,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.94099999999999995,"15min":0.92900000000000005},"min":{"1min":0.71999999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.2789999999999999,"5min":1.331,"15min":1.331},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":17.199000000000002,"15min":6.3840000000000003},"min":{"1min":0.66600000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1200000000000001,"5min":897.03999999999996,"15min":897.03999999999996},"last":1.1120000000000001}]},{"osd":2,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":18.686,"15min":36.369999999999997},"min":{"1min":0.746,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.1859999999999999,"5min":883.96900000000005,"15min":883.96900000000005},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.097,"15min":1.1180000000000001},"min":{"1min":0.81999999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.2609999999999999,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.96999999999999997}]},{"osd":3,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.069,"15min":1.085},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.232,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":6.298,"5min":2.141,"15min":1.45},"min":{"1min":0.80100000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":100.221,"5min":100.221,"15min":100.221},"last":1.0780000000000001}]},{"osd":4,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":3.6760000000000002,"15min":1.8700000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2869999999999999,"5min":272.95100000000002,"15min":272.95100000000002},"last":0.755},{"interface":"front","average":{"1min":1.081,"5min":1.1060000000000001,"15min":1.1539999999999999},"min":{"1min":0.79300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3360000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.94899999999999995}]},{"osd":6,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":4.0129999999999999,"15min":2.0590000000000002},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.28,"5min":309.56200000000001,"15min":309.56200000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2450000000000001,"5min":3.254,"15min":1.8680000000000001},"min":{"1min":0.90800000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.415,"5min":216.988,"15min":216.988},"last":0.88700000000000001}]},{"osd":7,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":7.7210000000000001,"15min":3.3849999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3680000000000001,"5min":609.06899999999996,"15min":609.06899999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":10.128,"15min":19.038},"min":{"1min":1.012,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.496,"5min":446.64499999999998,"15min":446.64499999999998},"last":1.0469999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5668,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366172,"statfs":{"total":96636764160,"available":96630960128,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":10.25,"15min":19.495999999999999},"min":{"1min":0.753,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.3,"5min":418.61799999999999,"15min":418.61799999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.085,"15min":1.0529999999999999},"min":{"1min":0.82799999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.411,"5min":1.48,"15min":1.48},"last":1.0109999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.016,"15min":1.022},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.296,"5min":1.421,"15min":1.421},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.153,"5min":1.155,"15min":1.1499999999999999},"min":{"1min":0.872,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.321,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1180000000000001}]},{"osd":2,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.98099999999999998,"15min":0.93600000000000005},"min":{"1min":0.754,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.071},{"interface":"front","average":{"1min":1.1459999999999999,"5min":12.583,"15min":10.872},"min":{"1min":0.88600000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.665,"5min":633.17700000000002,"15min":633.17700000000002},"last":1.161}]},{"osd":3,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":27.902000000000001,"15min":43.999000000000002},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.244,"5min":918.53399999999999,"15min":918.53399999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0169999999999999,"15min":0.999},"min":{"1min":0.76000000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4139999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0489999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.1970000000000001,"15min":1.139},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":16.021999999999998,"5min":16.021999999999998,"15min":16.021999999999998},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.018,"15min":1.03},"min":{"1min":0.71099999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3029999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.94299999999999995}]},{"osd":6,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":9.1929999999999996,"15min":14.262},"min":{"1min":0.77300000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.478,"5min":299.57100000000003,"15min":299.57100000000003},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.175,"5min":1.2050000000000001,"15min":1.2090000000000001},"min":{"1min":0.76800000000000002,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.369,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.2430000000000001}]},{"osd":7,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":4.8760000000000003,"5min":1.8939999999999999,"15min":1.415},"min":{"1min":0.78900000000000003,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":88.852000000000004,"5min":88.852000000000004,"15min":88.852000000000004},"last":1.097},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.181,"15min":1.194},"min":{"1min":0.82199999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.6259999999999999,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.82799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5668,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366172,"statfs":{"total":96636764160,"available":96630960128,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.71699999999999997},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76000000000000001,"15min":0.748},"min":{"1min":0.52300000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0249999999999999,"5min":1.073,"15min":1.073},"last":0.78200000000000003}]},{"osd":1,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":49.259,"5min":10.503,"15min":4.0309999999999997},"min":{"1min":0.64700000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":873.54200000000003,"5min":873.54200000000003,"15min":873.54200000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.85699999999999998,"15min":0.86499999999999999},"min":{"1min":0.59099999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.98499999999999999,"5min":1.607,"15min":1.607},"last":1.26}]},{"osd":2,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.89100000000000001,"15min":0.89600000000000002},"min":{"1min":0.66800000000000004,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1819999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91300000000000003,"15min":0.90300000000000002},"min":{"1min":0.72499999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.3220000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.05}]},{"osd":4,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":15.112,"15min":5.7560000000000002},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.115,"5min":735.82799999999997,"15min":735.82799999999997},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0960000000000001,"15min":1.1459999999999999},"min":{"1min":0.84999999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.24,"5min":1.875,"15min":1.875},"last":0.93700000000000006}]},{"osd":5,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":4.9619999999999997,"15min":2.4500000000000002},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.3480000000000001,"5min":363.78300000000002,"15min":363.78300000000002},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.1339999999999999,"15min":1.1639999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.226,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.1180000000000001}]},{"osd":6,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.1479999999999999,"15min":1.1819999999999999},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.321,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.038},{"interface":"front","average":{"1min":25.475999999999999,"5min":6.04,"15min":2.8239999999999998},"min":{"1min":0.94599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":462.90600000000001,"5min":462.90600000000001,"15min":462.90600000000001},"last":1.333}]},{"osd":7,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":18.411999999999999,"15min":6.9260000000000002},"min":{"1min":0.94299999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.3660000000000001,"5min":553.26900000000001,"15min":553.26900000000001},"last":1.161},{"interface":"front","average":{"1min":1.1379999999999999,"5min":19.024999999999999,"15min":27.608000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.399,"5min":583.47299999999996,"15min":583.47299999999996},"last":1.2290000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5728,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366112,"statfs":{"total":96636764160,"available":96630898688,"internally_reserved":0,"allocated":483328,"data_stored":192859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":3.1920000000000002,"15min":1.5609999999999999},"min":{"1min":0.53700000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.504,"5min":272.59399999999999,"15min":272.59399999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":6.5250000000000004,"15min":12.198},"min":{"1min":0.60199999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.206,"5min":298.29399999999998,"15min":298.29399999999998},"last":0.76400000000000001}]},{"osd":1,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":2.746,"15min":1.4379999999999999},"min":{"1min":0.54600000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.294,"5min":217.208,"15min":217.208},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":11.891,"15min":4.5350000000000001},"min":{"1min":0.68999999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2989999999999999,"5min":771.41800000000001,"15min":771.41800000000001},"last":0.85999999999999999}]},{"osd":3,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89100000000000001,"15min":0.93200000000000005},"min":{"1min":0.69099999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.028,"5min":1.782,"15min":1.782},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":5.6840000000000002,"15min":2.5640000000000001},"min":{"1min":0.72899999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8759999999999999,"5min":519.80100000000004,"15min":519.80100000000004},"last":0.99299999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.0609999999999999,"15min":1.087},"min":{"1min":0.78600000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":34.162999999999997,"5min":8.1120000000000001,"15min":3.4460000000000002},"min":{"1min":0.80100000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":501.62200000000001,"5min":501.62200000000001,"15min":501.62200000000001},"last":1.0409999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0569999999999999,"15min":1.079},"min":{"1min":0.752,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.3169999999999999,"5min":1.895,"15min":1.895},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.123,"5min":4.1070000000000002,"15min":2.1309999999999998},"min":{"1min":0.89400000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.9930000000000001,"5min":359.505,"15min":359.505},"last":0.94299999999999995}]},{"osd":6,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":8.6430000000000007,"15min":3.6850000000000001},"min":{"1min":0.84699999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.472,"5min":786.60400000000004,"15min":786.60400000000004},"last":1.069},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.84199999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.9410000000000001,"5min":1.944,"15min":1.944},"last":1.131}]},{"osd":7,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.238},"min":{"1min":0.88,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0310000000000001,"5min":2.048,"15min":2.048},"last":1.0920000000000001},{"interface":"front","average":{"1min":7.0119999999999996,"5min":2.3420000000000001,"15min":1.5940000000000001},"min":{"1min":0.98299999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":111.39400000000001,"5min":111.39400000000001,"15min":111.39400000000001},"last":1.163}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5732,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366108,"statfs":{"total":96636764160,"available":96630894592,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":9.9900000000000002,"15min":16.190000000000001},"min":{"1min":0.53700000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.321,"5min":553.65599999999995,"15min":553.65599999999995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":6.5229999999999997,"15min":2.7650000000000001},"min":{"1min":0.65800000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.087,"5min":501.34699999999998,"15min":501.34699999999998},"last":0.73099999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":4.0469999999999997,"15min":1.865},"min":{"1min":0.57499999999999996,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91300000000000003,"5min":313.49000000000001,"15min":313.49000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.86099999999999999,"15min":0.84299999999999997},"min":{"1min":0.60499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.282,"5min":1.319,"15min":1.319},"last":0.97199999999999998}]},{"osd":3,"last update":"Sat Sep 30 01:01:01 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.92400000000000004,"15min":0.91200000000000003},"min":{"1min":0.747,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.014},{"interface":"front","average":{"1min":55.460999999999999,"5min":31.306000000000001,"15min":11.065},"min":{"1min":0.78300000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":873.47900000000004,"5min":873.47900000000004,"15min":873.47900000000004},"last":873.47900000000004}]},{"osd":4,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.042,"15min":1.0880000000000001},"min":{"1min":0.76100000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.502,"5min":1.556,"15min":1.556},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1160000000000001,"15min":1.1180000000000001},"min":{"1min":0.88700000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.177}]},{"osd":5,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0529999999999999,"15min":1.089},"min":{"1min":0.83099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5169999999999999,"5min":1.71,"15min":1.71},"last":1.083},{"interface":"front","average":{"1min":37.758000000000003,"5min":10.032999999999999,"15min":6.3899999999999997},"min":{"1min":0.85499999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":586.91999999999996,"5min":586.91999999999996,"15min":586.91999999999996},"last":1.1319999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.123,"15min":1.161},"min":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.464,"5min":1.77,"15min":1.77},"last":1.246},{"interface":"front","average":{"1min":1.2110000000000001,"5min":6.8920000000000003,"15min":3.0840000000000001},"min":{"1min":1.0920000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.306,"5min":544.41600000000005,"15min":544.41600000000005},"last":1.28}]},{"osd":7,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.173,"15min":1.163},"min":{"1min":1.016,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.49,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.234,"5min":1.21,"15min":1.242},"min":{"1min":1.0549999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4279999999999999,"5min":1.861,"15min":1.861},"last":1.3360000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6240,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365600,"statfs":{"total":96636764160,"available":96630374400,"internally_reserved":0,"allocated":548864,"data_stored":722743,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":7.1059999999999999,"15min":9.6199999999999992},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.99299999999999999,"5min":309.20299999999997,"15min":309.20299999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":5.4189999999999996,"15min":2.3159999999999998},"min":{"1min":0.60799999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98099999999999998,"5min":533.17499999999995,"15min":533.17499999999995},"last":0.72599999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":14.385999999999999,"15min":27.942},"min":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0229999999999999,"5min":676.57399999999996,"15min":676.57399999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":24.475000000000001,"5min":18.829000000000001,"15min":6.859},"min":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":497.19900000000001,"5min":898.26700000000005,"15min":898.26700000000005},"last":0.78300000000000003}]},{"osd":3,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":4.0250000000000004,"15min":1.897},"min":{"1min":0.70599999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.214,"5min":362.38999999999999,"15min":362.38999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":1.0109999999999999},"min":{"1min":0.73499999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3360000000000001,"5min":1.585,"15min":1.585},"last":0.83599999999999997}]},{"osd":4,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.05,"15min":1.0569999999999999},"min":{"1min":0.90700000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.1919999999999999,"5min":1.377,"15min":1.377},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.117,"15min":1.1539999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.21,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.91200000000000003}]},{"osd":5,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":12.81,"15min":5.0179999999999998},"min":{"1min":0.79600000000000004,"5min":0.75,"15min":0.75},"max":{"1min":1.4059999999999999,"5min":667.67700000000002,"15min":667.67700000000002},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.107,"5min":4.2110000000000003,"15min":2.1120000000000001},"min":{"1min":0.97599999999999998,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.367,"5min":360.90100000000001,"15min":360.90100000000001},"last":1.232}]},{"osd":6,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":8.8379999999999992,"5min":2.6869999999999998,"15min":1.72},"min":{"1min":0.88900000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":163.465,"5min":163.465,"15min":163.465},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.05,"5min":14.26,"15min":5.5599999999999996},"min":{"1min":0.877,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.3540000000000001,"5min":970.15200000000004,"15min":970.15200000000004},"last":1.0409999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:00:04 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":7.8319999999999999,"15min":7.5570000000000004},"min":{"1min":0.88900000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.367,"5min":501.57299999999998,"15min":501.57299999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":22.666,"5min":13.086,"15min":5.1689999999999996},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":452.51799999999997,"5min":501.84800000000001,"15min":501.84800000000001},"last":1.2050000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:01:03.499 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-30T01:01:03.499 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:01:03.789 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:01:03.792 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-09-30T01:01:03.803 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2023-09-30T01:01:03.089552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":47184,"kb_used_data":3984,"kb_used_omap":0,"kb_used_meta":43136,"kb_avail":754927536,"statfs":{"total":773094113280,"available":773045796864,"internally_reserved":0,"allocated":4079616,"data_stored":3136348,"data_compressed":89871,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":44171264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.602802"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.280356+0000","last_change":"2023-09-30T01:00:02.106924+0000","last_active":"2023-09-30T01:00:45.280356+0000","last_peered":"2023-09-30T01:00:45.280356+0000","last_clean":"2023-09-30T01:00:45.280356+0000","last_became_active":"2023-09-30T00:55:55.343702+0000","last_became_peered":"2023-09-30T00:55:55.343702+0000","last_unstale":"2023-09-30T01:00:45.280356+0000","last_undegraded":"2023-09-30T01:00:45.280356+0000","last_fullsized":"2023-09-30T01:00:45.280356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:02.106695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:11.864778+0000","last_clean_scrub_stamp":"2023-09-30T01:00:02.106695+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:11.197424+0000","scrub_duration":0.0047035439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.992512+0000","last_change":"2023-09-30T01:00:00.083018+0000","last_active":"2023-09-30T01:00:45.992512+0000","last_peered":"2023-09-30T01:00:45.992512+0000","last_clean":"2023-09-30T01:00:45.992512+0000","last_became_active":"2023-09-30T00:55:55.372455+0000","last_became_peered":"2023-09-30T00:55:55.372455+0000","last_unstale":"2023-09-30T01:00:45.992512+0000","last_undegraded":"2023-09-30T01:00:45.992512+0000","last_fullsized":"2023-09-30T01:00:45.992512+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:00.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T01:00:00.082925+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:05.167676+0000","scrub_duration":0.0039526120000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":64,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993907+0000","last_change":"2023-09-30T00:59:52.818842+0000","last_active":"2023-09-30T01:00:45.993907+0000","last_peered":"2023-09-30T01:00:45.993907+0000","last_clean":"2023-09-30T01:00:45.993907+0000","last_became_active":"2023-09-30T00:55:55.371497+0000","last_became_peered":"2023-09-30T00:55:55.371497+0000","last_unstale":"2023-09-30T01:00:45.993907+0000","last_undegraded":"2023-09-30T01:00:45.993907+0000","last_fullsized":"2023-09-30T01:00:45.993907+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:52.818607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:59:52.818607+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:15.647590+0000","scrub_duration":0.0043827509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.992377+0000","last_change":"2023-09-30T01:00:11.015444+0000","last_active":"2023-09-30T01:00:45.992377+0000","last_peered":"2023-09-30T01:00:45.992377+0000","last_clean":"2023-09-30T01:00:45.992377+0000","last_became_active":"2023-09-30T00:55:55.350964+0000","last_became_peered":"2023-09-30T00:55:55.350964+0000","last_unstale":"2023-09-30T01:00:45.992377+0000","last_undegraded":"2023-09-30T01:00:45.992377+0000","last_fullsized":"2023-09-30T01:00:45.992377+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:11.015207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T01:00:11.015207+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:38.038876+0000","scrub_duration":0.0042614180000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993012+0000","last_change":"2023-09-30T00:59:56.922793+0000","last_active":"2023-09-30T01:00:45.993012+0000","last_peered":"2023-09-30T01:00:45.993012+0000","last_clean":"2023-09-30T01:00:45.993012+0000","last_became_active":"2023-09-30T00:55:55.342131+0000","last_became_peered":"2023-09-30T00:55:55.342131+0000","last_unstale":"2023-09-30T01:00:45.993012+0000","last_undegraded":"2023-09-30T01:00:45.993012+0000","last_fullsized":"2023-09-30T01:00:45.993012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:56.922501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:06.562848+0000","last_clean_scrub_stamp":"2023-09-30T00:59:56.922501+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:24.747687+0000","scrub_duration":0.0040857139999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.277663+0000","last_change":"2023-09-30T00:59:48.922555+0000","last_active":"2023-09-30T01:00:45.277663+0000","last_peered":"2023-09-30T01:00:45.277663+0000","last_clean":"2023-09-30T01:00:45.277663+0000","last_became_active":"2023-09-30T00:55:55.350971+0000","last_became_peered":"2023-09-30T00:55:55.350971+0000","last_unstale":"2023-09-30T01:00:45.277663+0000","last_undegraded":"2023-09-30T01:00:45.277663+0000","last_fullsized":"2023-09-30T01:00:45.277663+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:48.922299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:58:20.471763+0000","last_clean_scrub_stamp":"2023-09-30T00:59:48.922299+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:11.800573+0000","scrub_duration":0.0043283000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":64,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993821+0000","last_change":"2023-09-30T00:59:49.817759+0000","last_active":"2023-09-30T01:00:45.993821+0000","last_peered":"2023-09-30T01:00:45.993821+0000","last_clean":"2023-09-30T01:00:45.993821+0000","last_became_active":"2023-09-30T00:55:55.350574+0000","last_became_peered":"2023-09-30T00:55:55.350574+0000","last_unstale":"2023-09-30T01:00:45.993821+0000","last_undegraded":"2023-09-30T01:00:45.993821+0000","last_fullsized":"2023-09-30T01:00:45.993821+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:49.817496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:59:49.817496+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:11.184156+0000","scrub_duration":0.0079735229999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993084+0000","last_change":"2023-09-30T00:59:37.953173+0000","last_active":"2023-09-30T01:00:45.993084+0000","last_peered":"2023-09-30T01:00:45.993084+0000","last_clean":"2023-09-30T01:00:45.993084+0000","last_became_active":"2023-09-30T00:55:55.351023+0000","last_became_peered":"2023-09-30T00:55:55.351023+0000","last_unstale":"2023-09-30T01:00:45.993084+0000","last_undegraded":"2023-09-30T01:00:45.993084+0000","last_fullsized":"2023-09-30T01:00:45.993084+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:37.952950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:13.467244+0000","last_clean_scrub_stamp":"2023-09-30T00:59:37.952950+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:07.858959+0000","scrub_duration":0.0040933410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":216,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-09-30T01:00:45.993903+0000","last_change":"2023-09-30T01:00:04.742088+0000","last_active":"2023-09-30T01:00:45.993903+0000","last_peered":"2023-09-30T01:00:45.993903+0000","last_clean":"2023-09-30T01:00:45.993903+0000","last_became_active":"2023-09-30T00:55:54.335401+0000","last_became_peered":"2023-09-30T00:55:54.335401+0000","last_unstale":"2023-09-30T01:00:45.993903+0000","last_undegraded":"2023-09-30T01:00:45.993903+0000","last_fullsized":"2023-09-30T01:00:45.993903+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'117","last_scrub_stamp":"2023-09-30T01:00:04.741859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:53.298069+0000","last_clean_scrub_stamp":"2023-09-30T01:00:04.741859+0000","objects_scrubbed":2,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:16.702808+0000","scrub_duration":0.0048182659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":1586784,"data_compressed":89871,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6240,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365600,"statfs":{"total":96636764160,"available":96630374400,"internally_reserved":0,"allocated":548864,"data_stored":722743,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":1.054,"15min":1.514},"min":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.135,"5min":18.013999999999999,"15min":18.013999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.127,"15min":1.6519999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.145,"5min":20.856999999999999,"15min":20.856999999999999},"last":0.873}]},{"osd":1,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.849,"15min":4.306},"min":{"1min":0.63800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.24,"5min":50.606999999999999,"15min":50.606999999999999},"last":1.01},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.357,"15min":2.4209999999999998},"min":{"1min":0.70699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4199999999999999,"5min":41.756999999999998,"15min":41.756999999999998},"last":0.72899999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.1739999999999999,"15min":1.641},"min":{"1min":0.83699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.1080000000000001,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":23.733000000000001,"5min":8.5530000000000008,"15min":4.1509999999999998},"min":{"1min":0.86899999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":364.00400000000002,"5min":364.00400000000002,"15min":364.00400000000002},"last":0.94899999999999995}]},{"osd":3,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":39.890000000000001,"5min":9.0020000000000007,"15min":4.1059999999999999},"min":{"1min":0.626,"5min":0.621,"15min":0.621},"max":{"1min":501.76100000000002,"5min":501.76100000000002,"15min":501.76100000000002},"last":1.101},{"interface":"front","average":{"1min":30.462,"5min":7.0800000000000001,"15min":3.5630000000000002},"min":{"1min":0.80800000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":501.42500000000001,"5min":501.42500000000001,"15min":501.42500000000001},"last":0.77100000000000002}]},{"osd":4,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.03,"15min":1.1819999999999999},"min":{"1min":0.70399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.482,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":15.436,"15min":6.0030000000000001},"min":{"1min":0.748,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2170000000000001,"5min":829.03899999999999,"15min":829.03899999999999},"last":1.028}]},{"osd":5,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":8.3789999999999996,"15min":3.7149999999999999},"min":{"1min":0.84899999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.393,"5min":830.25300000000004,"15min":830.25300000000004},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":6.4749999999999996,"15min":3.1070000000000002},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.2330000000000001,"5min":530.12199999999996,"15min":530.12199999999996},"last":1.133}]},{"osd":6,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":9.6679999999999993,"15min":4.21},"min":{"1min":0.86699999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.3080000000000001,"5min":971.51400000000001,"15min":971.51400000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.2110000000000001,"15min":1.359},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.2370000000000001,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":0.98399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6240,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365600,"statfs":{"total":96636764160,"available":96630374400,"internally_reserved":0,"allocated":548864,"data_stored":722743,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":32.222999999999999,"5min":7.2439999999999998,"15min":3.0750000000000002},"min":{"1min":0.71999999999999997,"5min":0.627,"15min":0.627},"max":{"1min":500.60300000000001,"5min":500.60300000000001,"15min":500.60300000000001},"last":1.123},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.94699999999999995,"15min":0.94199999999999995},"min":{"1min":0.69699999999999995,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4470000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.095}]},{"osd":1,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":10.343999999999999,"15min":4.0540000000000003},"min":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.369,"5min":688.31500000000005,"15min":688.31500000000005},"last":1.02},{"interface":"front","average":{"1min":0.95799999999999996,"5min":5.7679999999999998,"15min":2.6120000000000001},"min":{"1min":0.69099999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.248,"5min":501.56200000000001,"15min":501.56200000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":2.9100000000000001,"15min":1.6399999999999999},"min":{"1min":0.72499999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3149999999999999,"5min":201.053,"15min":201.053},"last":1.204},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0049999999999999,"15min":0.98799999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.335,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.94999999999999996}]},{"osd":3,"last update":"Sat Sep 30 00:59:59 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":8.2810000000000006,"15min":3.4790000000000001},"min":{"1min":0.82799999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.3939999999999999,"5min":756.92499999999995,"15min":756.92499999999995},"last":1.161},{"interface":"front","average":{"1min":54.628,"5min":21.196000000000002,"15min":7.766},"min":{"1min":0.80200000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":859.12800000000004,"5min":859.12800000000004,"15min":859.12800000000004},"last":0.98099999999999998}]},{"osd":4,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":7.3449999999999998,"15min":3.117},"min":{"1min":0.754,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4139999999999999,"5min":358.13099999999997,"15min":358.13099999999997},"last":1.327},{"interface":"front","average":{"1min":1.0569999999999999,"5min":6.0759999999999996,"15min":2.7120000000000002},"min":{"1min":0.80100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.246,"5min":528.37900000000002,"15min":528.37900000000002},"last":1.2330000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":39.414000000000001,"5min":8.7230000000000008,"15min":3.589},"min":{"1min":0.96699999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":651.43100000000004,"5min":651.43100000000004,"15min":651.43100000000004},"last":1.363},{"interface":"front","average":{"1min":1.069,"5min":1.0900000000000001,"15min":1.087},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.3220000000000001,"5min":1.641,"15min":1.641},"last":0.91600000000000004}]},{"osd":7,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1759999999999999,"15min":1.1799999999999999},"min":{"1min":0.84999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.492,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.1850000000000001,"15min":1.181},"min":{"1min":1.0169999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.3049999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.069}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5668,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366172,"statfs":{"total":96636764160,"available":96630960128,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.018,"15min":1.052},"min":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2190000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.022,"15min":1.0409999999999999},"min":{"1min":0.79400000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2529999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.135}]},{"osd":1,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.94099999999999995,"15min":0.92900000000000005},"min":{"1min":0.71999999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.2789999999999999,"5min":1.331,"15min":1.331},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":17.199000000000002,"15min":6.3840000000000003},"min":{"1min":0.66600000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1200000000000001,"5min":897.03999999999996,"15min":897.03999999999996},"last":1.1120000000000001}]},{"osd":2,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":18.686,"15min":36.369999999999997},"min":{"1min":0.746,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.1859999999999999,"5min":883.96900000000005,"15min":883.96900000000005},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.097,"15min":1.1180000000000001},"min":{"1min":0.81999999999999995,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.2609999999999999,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.96999999999999997}]},{"osd":3,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.069,"15min":1.085},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.232,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":6.298,"5min":2.141,"15min":1.45},"min":{"1min":0.80100000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":100.221,"5min":100.221,"15min":100.221},"last":1.0780000000000001}]},{"osd":4,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":3.6760000000000002,"15min":1.8700000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2869999999999999,"5min":272.95100000000002,"15min":272.95100000000002},"last":0.755},{"interface":"front","average":{"1min":1.081,"5min":1.1060000000000001,"15min":1.1539999999999999},"min":{"1min":0.79300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3360000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.94899999999999995}]},{"osd":6,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":4.0129999999999999,"15min":2.0590000000000002},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.28,"5min":309.56200000000001,"15min":309.56200000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2450000000000001,"5min":3.254,"15min":1.8680000000000001},"min":{"1min":0.90800000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.415,"5min":216.988,"15min":216.988},"last":0.88700000000000001}]},{"osd":7,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":7.7210000000000001,"15min":3.3849999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3680000000000001,"5min":609.06899999999996,"15min":609.06899999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":10.128,"15min":19.038},"min":{"1min":1.012,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.496,"5min":446.64499999999998,"15min":446.64499999999998},"last":1.0469999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5668,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366172,"statfs":{"total":96636764160,"available":96630960128,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":10.25,"15min":19.495999999999999},"min":{"1min":0.753,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.3,"5min":418.61799999999999,"15min":418.61799999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.085,"15min":1.0529999999999999},"min":{"1min":0.82799999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.411,"5min":1.48,"15min":1.48},"last":1.0109999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.016,"15min":1.022},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.296,"5min":1.421,"15min":1.421},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.153,"5min":1.155,"15min":1.1499999999999999},"min":{"1min":0.872,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.321,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1180000000000001}]},{"osd":2,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.98099999999999998,"15min":0.93600000000000005},"min":{"1min":0.754,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.071},{"interface":"front","average":{"1min":1.1459999999999999,"5min":12.583,"15min":10.872},"min":{"1min":0.88600000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.665,"5min":633.17700000000002,"15min":633.17700000000002},"last":1.161}]},{"osd":3,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":27.902000000000001,"15min":43.999000000000002},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.244,"5min":918.53399999999999,"15min":918.53399999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0169999999999999,"15min":0.999},"min":{"1min":0.76000000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4139999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0489999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.1970000000000001,"15min":1.139},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":16.021999999999998,"5min":16.021999999999998,"15min":16.021999999999998},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.018,"15min":1.03},"min":{"1min":0.71099999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.3029999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.94299999999999995}]},{"osd":6,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":9.1929999999999996,"15min":14.262},"min":{"1min":0.77300000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.478,"5min":299.57100000000003,"15min":299.57100000000003},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.175,"5min":1.2050000000000001,"15min":1.2090000000000001},"min":{"1min":0.76800000000000002,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.369,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.2430000000000001}]},{"osd":7,"last update":"Sat Sep 30 01:00:01 2023","interfaces":[{"interface":"back","average":{"1min":4.8760000000000003,"5min":1.8939999999999999,"15min":1.415},"min":{"1min":0.78900000000000003,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":88.852000000000004,"5min":88.852000000000004,"15min":88.852000000000004},"last":1.097},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.181,"15min":1.194},"min":{"1min":0.82199999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.6259999999999999,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.82799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5668,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366172,"statfs":{"total":96636764160,"available":96630960128,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.71699999999999997},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76000000000000001,"15min":0.748},"min":{"1min":0.52300000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0249999999999999,"5min":1.073,"15min":1.073},"last":0.78200000000000003}]},{"osd":1,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":49.259,"5min":10.503,"15min":4.0309999999999997},"min":{"1min":0.64700000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":873.54200000000003,"5min":873.54200000000003,"15min":873.54200000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.85699999999999998,"15min":0.86499999999999999},"min":{"1min":0.59099999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.98499999999999999,"5min":1.607,"15min":1.607},"last":1.26}]},{"osd":2,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.89100000000000001,"15min":0.89600000000000002},"min":{"1min":0.66800000000000004,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1819999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91300000000000003,"15min":0.90300000000000002},"min":{"1min":0.72499999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.3220000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.05}]},{"osd":4,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":15.112,"15min":5.7560000000000002},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.115,"5min":735.82799999999997,"15min":735.82799999999997},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0960000000000001,"15min":1.1459999999999999},"min":{"1min":0.84999999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.24,"5min":1.875,"15min":1.875},"last":0.93700000000000006}]},{"osd":5,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":4.9619999999999997,"15min":2.4500000000000002},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.3480000000000001,"5min":363.78300000000002,"15min":363.78300000000002},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.1339999999999999,"15min":1.1639999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.226,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.1180000000000001}]},{"osd":6,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.1479999999999999,"15min":1.1819999999999999},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.321,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.038},{"interface":"front","average":{"1min":25.475999999999999,"5min":6.04,"15min":2.8239999999999998},"min":{"1min":0.94599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":462.90600000000001,"5min":462.90600000000001,"15min":462.90600000000001},"last":1.333}]},{"osd":7,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":18.411999999999999,"15min":6.9260000000000002},"min":{"1min":0.94299999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.3660000000000001,"5min":553.26900000000001,"15min":553.26900000000001},"last":1.161},{"interface":"front","average":{"1min":1.1379999999999999,"5min":19.024999999999999,"15min":27.608000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.399,"5min":583.47299999999996,"15min":583.47299999999996},"last":1.2290000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5728,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366112,"statfs":{"total":96636764160,"available":96630898688,"internally_reserved":0,"allocated":483328,"data_stored":192859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":3.1920000000000002,"15min":1.5609999999999999},"min":{"1min":0.53700000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.504,"5min":272.59399999999999,"15min":272.59399999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":6.5250000000000004,"15min":12.198},"min":{"1min":0.60199999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.206,"5min":298.29399999999998,"15min":298.29399999999998},"last":0.76400000000000001}]},{"osd":1,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":2.746,"15min":1.4379999999999999},"min":{"1min":0.54600000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.294,"5min":217.208,"15min":217.208},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":11.891,"15min":4.5350000000000001},"min":{"1min":0.68999999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2989999999999999,"5min":771.41800000000001,"15min":771.41800000000001},"last":0.85999999999999999}]},{"osd":3,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89100000000000001,"15min":0.93200000000000005},"min":{"1min":0.69099999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.028,"5min":1.782,"15min":1.782},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":5.6840000000000002,"15min":2.5640000000000001},"min":{"1min":0.72899999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8759999999999999,"5min":519.80100000000004,"15min":519.80100000000004},"last":0.99299999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.0609999999999999,"15min":1.087},"min":{"1min":0.78600000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":34.162999999999997,"5min":8.1120000000000001,"15min":3.4460000000000002},"min":{"1min":0.80100000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":501.62200000000001,"5min":501.62200000000001,"15min":501.62200000000001},"last":1.0409999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0569999999999999,"15min":1.079},"min":{"1min":0.752,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.3169999999999999,"5min":1.895,"15min":1.895},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.123,"5min":4.1070000000000002,"15min":2.1309999999999998},"min":{"1min":0.89400000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.9930000000000001,"5min":359.505,"15min":359.505},"last":0.94299999999999995}]},{"osd":6,"last update":"Sat Sep 30 00:59:58 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":8.6430000000000007,"15min":3.6850000000000001},"min":{"1min":0.84699999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.472,"5min":786.60400000000004,"15min":786.60400000000004},"last":1.069},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2130000000000001},"min":{"1min":0.84199999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.9410000000000001,"5min":1.944,"15min":1.944},"last":1.131}]},{"osd":7,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.238},"min":{"1min":0.88,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0310000000000001,"5min":2.048,"15min":2.048},"last":1.0920000000000001},{"interface":"front","average":{"1min":7.0119999999999996,"5min":2.3420000000000001,"15min":1.5940000000000001},"min":{"1min":0.98299999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":111.39400000000001,"5min":111.39400000000001,"15min":111.39400000000001},"last":1.163}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5732,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366108,"statfs":{"total":96636764160,"available":96630894592,"internally_reserved":0,"allocated":487424,"data_stored":193815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":9.9900000000000002,"15min":16.190000000000001},"min":{"1min":0.53700000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.321,"5min":553.65599999999995,"15min":553.65599999999995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":6.5229999999999997,"15min":2.7650000000000001},"min":{"1min":0.65800000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.087,"5min":501.34699999999998,"15min":501.34699999999998},"last":0.73099999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":4.0469999999999997,"15min":1.865},"min":{"1min":0.57499999999999996,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91300000000000003,"5min":313.49000000000001,"15min":313.49000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.86099999999999999,"15min":0.84299999999999997},"min":{"1min":0.60499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.282,"5min":1.319,"15min":1.319},"last":0.97199999999999998}]},{"osd":3,"last update":"Sat Sep 30 01:01:01 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.92400000000000004,"15min":0.91200000000000003},"min":{"1min":0.747,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.014},{"interface":"front","average":{"1min":55.460999999999999,"5min":31.306000000000001,"15min":11.065},"min":{"1min":0.78300000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":873.47900000000004,"5min":873.47900000000004,"15min":873.47900000000004},"last":873.47900000000004}]},{"osd":4,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.042,"15min":1.0880000000000001},"min":{"1min":0.76100000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.502,"5min":1.556,"15min":1.556},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1160000000000001,"15min":1.1180000000000001},"min":{"1min":0.88700000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.177}]},{"osd":5,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0529999999999999,"15min":1.089},"min":{"1min":0.83099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5169999999999999,"5min":1.71,"15min":1.71},"last":1.083},{"interface":"front","average":{"1min":37.758000000000003,"5min":10.032999999999999,"15min":6.3899999999999997},"min":{"1min":0.85499999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":586.91999999999996,"5min":586.91999999999996,"15min":586.91999999999996},"last":1.1319999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.123,"15min":1.161},"min":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.464,"5min":1.77,"15min":1.77},"last":1.246},{"interface":"front","average":{"1min":1.2110000000000001,"5min":6.8920000000000003,"15min":3.0840000000000001},"min":{"1min":1.0920000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.306,"5min":544.41600000000005,"15min":544.41600000000005},"last":1.28}]},{"osd":7,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.173,"15min":1.163},"min":{"1min":1.016,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.49,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.234,"5min":1.21,"15min":1.242},"min":{"1min":1.0549999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4279999999999999,"5min":1.861,"15min":1.861},"last":1.3360000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6240,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365600,"statfs":{"total":96636764160,"available":96630374400,"internally_reserved":0,"allocated":548864,"data_stored":722743,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":7.1059999999999999,"15min":9.6199999999999992},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.99299999999999999,"5min":309.20299999999997,"15min":309.20299999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":5.4189999999999996,"15min":2.3159999999999998},"min":{"1min":0.60799999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98099999999999998,"5min":533.17499999999995,"15min":533.17499999999995},"last":0.72599999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":14.385999999999999,"15min":27.942},"min":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0229999999999999,"5min":676.57399999999996,"15min":676.57399999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":24.475000000000001,"5min":18.829000000000001,"15min":6.859},"min":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":497.19900000000001,"5min":898.26700000000005,"15min":898.26700000000005},"last":0.78300000000000003}]},{"osd":3,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":4.0250000000000004,"15min":1.897},"min":{"1min":0.70599999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.214,"5min":362.38999999999999,"15min":362.38999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":1.0109999999999999},"min":{"1min":0.73499999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3360000000000001,"5min":1.585,"15min":1.585},"last":0.83599999999999997}]},{"osd":4,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.05,"15min":1.0569999999999999},"min":{"1min":0.90700000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.1919999999999999,"5min":1.377,"15min":1.377},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.117,"15min":1.1539999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.21,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.91200000000000003}]},{"osd":5,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":12.81,"15min":5.0179999999999998},"min":{"1min":0.79600000000000004,"5min":0.75,"15min":0.75},"max":{"1min":1.4059999999999999,"5min":667.67700000000002,"15min":667.67700000000002},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.107,"5min":4.2110000000000003,"15min":2.1120000000000001},"min":{"1min":0.97599999999999998,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.367,"5min":360.90100000000001,"15min":360.90100000000001},"last":1.232}]},{"osd":6,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":8.8379999999999992,"5min":2.6869999999999998,"15min":1.72},"min":{"1min":0.88900000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":163.465,"5min":163.465,"15min":163.465},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.05,"5min":14.26,"15min":5.5599999999999996},"min":{"1min":0.877,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.3540000000000001,"5min":970.15200000000004,"15min":970.15200000000004},"last":1.0409999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:00:04 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":7.8319999999999999,"15min":7.5570000000000004},"min":{"1min":0.88900000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.367,"5min":501.57299999999998,"15min":501.57299999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":22.666,"5min":13.086,"15min":5.1689999999999996},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":452.51799999999997,"5min":501.84800000000001,"15min":501.84800000000001},"last":1.2050000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:01:03.805 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T01:01:04.098 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:01:04.098 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":44,"fsid":"a4f1df1b-c238-45bb-b595-c54d4bb08739","created":"2023-09-30T00:55:42.211215+0000","modified":"2023-09-30T01:01:03.080432+0000","last_up_change":"2023-09-30T00:55:51.275589+0000","last_in_change":"2023-09-30T00:55:44.646942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T00:55:52.460922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T00:55:53.353727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3644de55-4abe-484a-ab81-37b6248b978c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":31243},{"type":"v1","addr":"172.21.15.69:6817","nonce":31243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":31243},{"type":"v1","addr":"172.21.15.69:6819","nonce":31243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":31243},{"type":"v1","addr":"172.21.15.69:6823","nonce":31243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":31243},{"type":"v1","addr":"172.21.15.69:6821","nonce":31243}]},"public_addr":"172.21.15.69:6817/31243","cluster_addr":"172.21.15.69:6819/31243","heartbeat_back_addr":"172.21.15.69:6823/31243","heartbeat_front_addr":"172.21.15.69:6821/31243","state":["exists","up"]},{"osd":1,"uuid":"e81ee49e-57c0-4bd4-81e1-943a53e17e0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":31242},{"type":"v1","addr":"172.21.15.69:6825","nonce":31242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":31242},{"type":"v1","addr":"172.21.15.69:6827","nonce":31242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":31242},{"type":"v1","addr":"172.21.15.69:6831","nonce":31242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":31242},{"type":"v1","addr":"172.21.15.69:6829","nonce":31242}]},"public_addr":"172.21.15.69:6825/31242","cluster_addr":"172.21.15.69:6827/31242","heartbeat_back_addr":"172.21.15.69:6831/31242","heartbeat_front_addr":"172.21.15.69:6829/31242","state":["exists","up"]},{"osd":2,"uuid":"95f49092-0cb9-4fcf-8ec8-740061c438c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":31245},{"type":"v1","addr":"172.21.15.69:6809","nonce":31245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":31245},{"type":"v1","addr":"172.21.15.69:6811","nonce":31245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":31245},{"type":"v1","addr":"172.21.15.69:6815","nonce":31245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":31245},{"type":"v1","addr":"172.21.15.69:6813","nonce":31245}]},"public_addr":"172.21.15.69:6809/31245","cluster_addr":"172.21.15.69:6811/31245","heartbeat_back_addr":"172.21.15.69:6815/31245","heartbeat_front_addr":"172.21.15.69:6813/31245","state":["exists","up"]},{"osd":3,"uuid":"22d0daa1-283e-477d-8088-31922a3ff54f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":31241},{"type":"v1","addr":"172.21.15.69:6801","nonce":31241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":31241},{"type":"v1","addr":"172.21.15.69:6803","nonce":31241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":31241},{"type":"v1","addr":"172.21.15.69:6807","nonce":31241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":31241},{"type":"v1","addr":"172.21.15.69:6805","nonce":31241}]},"public_addr":"172.21.15.69:6801/31241","cluster_addr":"172.21.15.69:6803/31241","heartbeat_back_addr":"172.21.15.69:6807/31241","heartbeat_front_addr":"172.21.15.69:6805/31241","state":["exists","up"]},{"osd":4,"uuid":"601a722e-5cd4-46b0-9651-74d99ab73be6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":31269},{"type":"v1","addr":"172.21.15.150:6801","nonce":31269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":31269},{"type":"v1","addr":"172.21.15.150:6803","nonce":31269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":31269},{"type":"v1","addr":"172.21.15.150:6807","nonce":31269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":31269},{"type":"v1","addr":"172.21.15.150:6805","nonce":31269}]},"public_addr":"172.21.15.150:6801/31269","cluster_addr":"172.21.15.150:6803/31269","heartbeat_back_addr":"172.21.15.150:6807/31269","heartbeat_front_addr":"172.21.15.150:6805/31269","state":["exists","up"]},{"osd":5,"uuid":"c726bca0-dcd7-4041-88b3-6234ddb84477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":31270},{"type":"v1","addr":"172.21.15.150:6809","nonce":31270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":31270},{"type":"v1","addr":"172.21.15.150:6811","nonce":31270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":31270},{"type":"v1","addr":"172.21.15.150:6815","nonce":31270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":31270},{"type":"v1","addr":"172.21.15.150:6813","nonce":31270}]},"public_addr":"172.21.15.150:6809/31270","cluster_addr":"172.21.15.150:6811/31270","heartbeat_back_addr":"172.21.15.150:6815/31270","heartbeat_front_addr":"172.21.15.150:6813/31270","state":["exists","up"]},{"osd":6,"uuid":"9dc1bc9b-177d-448c-8917-2c400d1d3ca4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":31271},{"type":"v1","addr":"172.21.15.150:6825","nonce":31271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":31271},{"type":"v1","addr":"172.21.15.150:6827","nonce":31271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":31271},{"type":"v1","addr":"172.21.15.150:6831","nonce":31271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":31271},{"type":"v1","addr":"172.21.15.150:6829","nonce":31271}]},"public_addr":"172.21.15.150:6825/31271","cluster_addr":"172.21.15.150:6827/31271","heartbeat_back_addr":"172.21.15.150:6831/31271","heartbeat_front_addr":"172.21.15.150:6829/31271","state":["exists","up"]},{"osd":7,"uuid":"45e5f524-aa7c-46a7-80a1-bfbeac544530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":31268},{"type":"v1","addr":"172.21.15.150:6817","nonce":31268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":31268},{"type":"v1","addr":"172.21.15.150:6819","nonce":31268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":31268},{"type":"v1","addr":"172.21.15.150:6823","nonce":31268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":31268},{"type":"v1","addr":"172.21.15.150:6821","nonce":31268}]},"public_addr":"172.21.15.150:6817/31268","cluster_addr":"172.21.15.150:6819/31268","heartbeat_back_addr":"172.21.15.150:6823/31268","heartbeat_front_addr":"172.21.15.150:6821/31268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:32.472034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:39.260646+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:22.573301+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:21.541272+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:01:02.466168+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T00:59:58.447620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:45.169826+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T01:00:49.927449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T01:01:05.114 INFO:tasks.ceph:Scrubbing osd.0 2023-09-30T01:01:05.114 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:01:05.285 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-30T01:01:05.285 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "" 2023-09-30T01:01:05.285 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-30T01:01:05.301 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-09-30T01:01:05.604 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 0 to deep-scrub 2023-09-30T01:01:05.614 INFO:tasks.ceph:Scrubbing osd.1 2023-09-30T01:01:05.614 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:01:05.769 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-30T01:01:05.769 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:01:05.770 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-30T01:01:05.782 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-09-30T01:01:06.085 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 1 to deep-scrub 2023-09-30T01:01:06.097 INFO:tasks.ceph:Scrubbing osd.2 2023-09-30T01:01:06.098 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:01:06.250 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-30T01:01:06.250 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:01:06.250 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-30T01:01:06.263 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-09-30T01:01:06.562 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 2 to deep-scrub 2023-09-30T01:01:06.577 INFO:tasks.ceph:Scrubbing osd.3 2023-09-30T01:01:06.577 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:01:06.725 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-30T01:01:06.725 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:01:06.725 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-30T01:01:06.737 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-09-30T01:01:07.043 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 3 to deep-scrub 2023-09-30T01:01:07.053 INFO:tasks.ceph:Scrubbing osd.4 2023-09-30T01:01:07.053 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:01:07.207 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-30T01:01:07.207 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:01:07.208 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-30T01:01:07.220 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-09-30T01:01:07.524 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 4 to deep-scrub 2023-09-30T01:01:07.535 INFO:tasks.ceph:Scrubbing osd.5 2023-09-30T01:01:07.535 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:01:07.693 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-30T01:01:07.693 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:01:07.693 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-30T01:01:07.707 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-09-30T01:01:08.006 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 5 to deep-scrub 2023-09-30T01:01:08.016 INFO:tasks.ceph:Scrubbing osd.6 2023-09-30T01:01:08.017 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:01:08.170 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-30T01:01:08.170 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:01:08.170 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-30T01:01:08.183 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-09-30T01:01:08.479 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 6 to deep-scrub 2023-09-30T01:01:08.489 INFO:tasks.ceph:Scrubbing osd.7 2023-09-30T01:01:08.489 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-09-30T01:01:08.644 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-30T01:01:08.644 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-30T01:01:08.644 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-30T01:01:08.657 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-09-30T01:01:08.964 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 7 to deep-scrub 2023-09-30T01:01:08.974 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:01:09.267 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:01:09.270 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-09-30T01:01:09.280 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2023-09-30T01:01:08.491177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":47504,"kb_used_data":4176,"kb_used_omap":0,"kb_used_meta":43264,"kb_avail":754927216,"statfs":{"total":773094113280,"available":773045469184,"internally_reserved":0,"allocated":4276224,"data_stored":3223516,"data_compressed":89871,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":44302336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002045"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-09-30T01:01:03.096643+0000","last_change":"2023-09-30T01:00:02.106924+0000","last_active":"2023-09-30T01:01:03.096643+0000","last_peered":"2023-09-30T01:01:03.096643+0000","last_clean":"2023-09-30T01:01:03.096643+0000","last_became_active":"2023-09-30T00:55:55.343702+0000","last_became_peered":"2023-09-30T00:55:55.343702+0000","last_unstale":"2023-09-30T01:01:03.096643+0000","last_undegraded":"2023-09-30T01:01:03.096643+0000","last_fullsized":"2023-09-30T01:01:03.096643+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:02.106695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:11.864778+0000","last_clean_scrub_stamp":"2023-09-30T01:00:02.106695+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:11.197424+0000","scrub_duration":0.0047035439999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:04.290363+0000","last_change":"2023-09-30T01:00:00.083018+0000","last_active":"2023-09-30T01:01:04.290363+0000","last_peered":"2023-09-30T01:01:04.290363+0000","last_clean":"2023-09-30T01:01:04.290363+0000","last_became_active":"2023-09-30T00:55:55.372455+0000","last_became_peered":"2023-09-30T00:55:55.372455+0000","last_unstale":"2023-09-30T01:01:04.290363+0000","last_undegraded":"2023-09-30T01:01:04.290363+0000","last_fullsized":"2023-09-30T01:01:04.290363+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:00.082925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T01:00:00.082925+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.0039526120000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":66,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-09-30T01:01:03.419016+0000","last_change":"2023-09-30T00:59:52.818842+0000","last_active":"2023-09-30T01:01:03.419016+0000","last_peered":"2023-09-30T01:01:03.419016+0000","last_clean":"2023-09-30T01:01:03.419016+0000","last_became_active":"2023-09-30T00:55:55.371497+0000","last_became_peered":"2023-09-30T00:55:55.371497+0000","last_unstale":"2023-09-30T01:01:03.419016+0000","last_undegraded":"2023-09-30T01:01:03.419016+0000","last_fullsized":"2023-09-30T01:01:03.419016+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:52.818607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:59:52.818607+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:15.647590+0000","scrub_duration":0.0043827509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:04.290360+0000","last_change":"2023-09-30T01:00:11.015444+0000","last_active":"2023-09-30T01:01:04.290360+0000","last_peered":"2023-09-30T01:01:04.290360+0000","last_clean":"2023-09-30T01:01:04.290360+0000","last_became_active":"2023-09-30T00:55:55.350964+0000","last_became_peered":"2023-09-30T00:55:55.350964+0000","last_unstale":"2023-09-30T01:01:04.290360+0000","last_undegraded":"2023-09-30T01:01:04.290360+0000","last_fullsized":"2023-09-30T01:01:04.290360+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:00:11.015207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T01:00:11.015207+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.0042614180000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":67,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-09-30T01:01:03.418455+0000","last_change":"2023-09-30T00:59:56.922793+0000","last_active":"2023-09-30T01:01:03.418455+0000","last_peered":"2023-09-30T01:01:03.418455+0000","last_clean":"2023-09-30T01:01:03.418455+0000","last_became_active":"2023-09-30T00:55:55.342131+0000","last_became_peered":"2023-09-30T00:55:55.342131+0000","last_unstale":"2023-09-30T01:01:03.418455+0000","last_undegraded":"2023-09-30T01:01:03.418455+0000","last_fullsized":"2023-09-30T01:01:03.418455+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:56.922501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:06.562848+0000","last_clean_scrub_stamp":"2023-09-30T00:59:56.922501+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:24.747687+0000","scrub_duration":0.0040857139999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:04.102595+0000","last_change":"2023-09-30T00:59:48.922555+0000","last_active":"2023-09-30T01:01:04.102595+0000","last_peered":"2023-09-30T01:01:04.102595+0000","last_clean":"2023-09-30T01:01:04.102595+0000","last_became_active":"2023-09-30T00:55:55.350971+0000","last_became_peered":"2023-09-30T00:55:55.350971+0000","last_unstale":"2023-09-30T01:01:04.102595+0000","last_undegraded":"2023-09-30T01:01:04.102595+0000","last_fullsized":"2023-09-30T01:01:04.102595+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:48.922299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:58:20.471763+0000","last_clean_scrub_stamp":"2023-09-30T00:59:48.922299+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.0043283000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":66,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-09-30T01:01:03.418954+0000","last_change":"2023-09-30T00:59:49.817759+0000","last_active":"2023-09-30T01:01:03.418954+0000","last_peered":"2023-09-30T01:01:03.418954+0000","last_clean":"2023-09-30T01:01:03.418954+0000","last_became_active":"2023-09-30T00:55:55.350574+0000","last_became_peered":"2023-09-30T00:55:55.350574+0000","last_unstale":"2023-09-30T01:01:03.418954+0000","last_undegraded":"2023-09-30T01:01:03.418954+0000","last_fullsized":"2023-09-30T01:01:03.418954+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:49.817496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:54.309464+0000","last_clean_scrub_stamp":"2023-09-30T00:59:49.817496+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:11.184156+0000","scrub_duration":0.0079735229999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":67,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-09-30T01:01:03.418429+0000","last_change":"2023-09-30T00:59:37.953173+0000","last_active":"2023-09-30T01:01:03.418429+0000","last_peered":"2023-09-30T01:01:03.418429+0000","last_clean":"2023-09-30T01:01:03.418429+0000","last_became_active":"2023-09-30T00:55:55.351023+0000","last_became_peered":"2023-09-30T00:55:55.351023+0000","last_unstale":"2023-09-30T01:01:03.418429+0000","last_undegraded":"2023-09-30T01:01:03.418429+0000","last_fullsized":"2023-09-30T01:01:03.418429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T00:59:37.952950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:57:13.467244+0000","last_clean_scrub_stamp":"2023-09-30T00:59:37.952950+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:07.858959+0000","scrub_duration":0.0040933410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":218,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-09-30T01:01:03.418965+0000","last_change":"2023-09-30T01:00:04.742088+0000","last_active":"2023-09-30T01:01:03.418965+0000","last_peered":"2023-09-30T01:01:03.418965+0000","last_clean":"2023-09-30T01:01:03.418965+0000","last_became_active":"2023-09-30T00:55:54.335401+0000","last_became_peered":"2023-09-30T00:55:54.335401+0000","last_unstale":"2023-09-30T01:01:03.418965+0000","last_undegraded":"2023-09-30T01:01:03.418965+0000","last_fullsized":"2023-09-30T01:01:03.418965+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'117","last_scrub_stamp":"2023-09-30T01:00:04.741859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T00:55:53.298069+0000","last_clean_scrub_stamp":"2023-09-30T01:00:04.741859+0000","objects_scrubbed":2,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:01:16.702808+0000","scrub_duration":0.0048182659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":1586784,"data_compressed":89871,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6264,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365576,"statfs":{"total":96636764160,"available":96630349824,"internally_reserved":0,"allocated":573440,"data_stored":733639,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":1.054,"15min":1.514},"min":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.135,"5min":18.013999999999999,"15min":18.013999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.127,"15min":1.6519999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.145,"5min":20.856999999999999,"15min":20.856999999999999},"last":1.0449999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.849,"15min":4.306},"min":{"1min":0.63800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.24,"5min":50.606999999999999,"15min":50.606999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.357,"15min":2.4209999999999998},"min":{"1min":0.70699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4199999999999999,"5min":41.756999999999998,"15min":41.756999999999998},"last":0.83999999999999997}]},{"osd":2,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.1739999999999999,"15min":1.641},"min":{"1min":0.83699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.1080000000000001,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":23.733000000000001,"5min":8.5530000000000008,"15min":4.1509999999999998},"min":{"1min":0.86899999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":364.00400000000002,"5min":364.00400000000002,"15min":364.00400000000002},"last":1.1619999999999999}]},{"osd":3,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":39.890000000000001,"5min":9.0020000000000007,"15min":4.1059999999999999},"min":{"1min":0.626,"5min":0.621,"15min":0.621},"max":{"1min":501.76100000000002,"5min":501.76100000000002,"15min":501.76100000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":30.462,"5min":7.0800000000000001,"15min":3.5630000000000002},"min":{"1min":0.80800000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":501.42500000000001,"5min":501.42500000000001,"15min":501.42500000000001},"last":1.083}]},{"osd":4,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.03,"15min":1.1819999999999999},"min":{"1min":0.70399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.482,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.127},{"interface":"front","average":{"1min":0.96099999999999997,"5min":15.436,"15min":6.0030000000000001},"min":{"1min":0.748,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2170000000000001,"5min":829.03899999999999,"15min":829.03899999999999},"last":0.95499999999999996}]},{"osd":5,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":8.3789999999999996,"15min":3.7149999999999999},"min":{"1min":0.84899999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.393,"5min":830.25300000000004,"15min":830.25300000000004},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":6.4749999999999996,"15min":3.1070000000000002},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.2330000000000001,"5min":530.12199999999996,"15min":530.12199999999996},"last":1.2430000000000001}]},{"osd":6,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":9.6679999999999993,"15min":4.21},"min":{"1min":0.86699999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.3080000000000001,"5min":971.51400000000001,"15min":971.51400000000001},"last":1.177},{"interface":"front","average":{"1min":1.085,"5min":1.2110000000000001,"15min":1.359},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.2370000000000001,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.034}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6264,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365576,"statfs":{"total":96636764160,"available":96630349824,"internally_reserved":0,"allocated":573440,"data_stored":733639,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":32.222999999999999,"5min":7.2439999999999998,"15min":3.0750000000000002},"min":{"1min":0.71999999999999997,"5min":0.627,"15min":0.627},"max":{"1min":500.60300000000001,"5min":500.60300000000001,"15min":500.60300000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.94699999999999995,"15min":0.94199999999999995},"min":{"1min":0.69699999999999995,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4470000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.0669999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":10.343999999999999,"15min":4.0540000000000003},"min":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.369,"5min":688.31500000000005,"15min":688.31500000000005},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.95799999999999996,"5min":5.7679999999999998,"15min":2.6120000000000001},"min":{"1min":0.69099999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.248,"5min":501.56200000000001,"15min":501.56200000000001},"last":0.72199999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":2.9100000000000001,"15min":1.6399999999999999},"min":{"1min":0.72499999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3149999999999999,"5min":201.053,"15min":201.053},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0049999999999999,"15min":0.98799999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.335,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.0149999999999999}]},{"osd":3,"last update":"Sat Sep 30 01:01:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":8.2810000000000006,"15min":3.4790000000000001},"min":{"1min":0.76900000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.304,"5min":756.92499999999995,"15min":756.92499999999995},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.012,"5min":21.189,"15min":7.7629999999999999},"min":{"1min":0.89400000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.159,"5min":859.12800000000004,"15min":859.12800000000004},"last":1.103}]},{"osd":4,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":7.3449999999999998,"15min":3.117},"min":{"1min":0.754,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4139999999999999,"5min":358.13099999999997,"15min":358.13099999999997},"last":0.872},{"interface":"front","average":{"1min":1.0569999999999999,"5min":6.0759999999999996,"15min":2.7120000000000002},"min":{"1min":0.80100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.246,"5min":528.37900000000002,"15min":528.37900000000002},"last":1.1279999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":39.414000000000001,"5min":8.7230000000000008,"15min":3.589},"min":{"1min":0.96699999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":651.43100000000004,"5min":651.43100000000004,"15min":651.43100000000004},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.069,"5min":1.0900000000000001,"15min":1.087},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.3220000000000001,"5min":1.641,"15min":1.641},"last":0.91800000000000004}]},{"osd":7,"last update":"Sat Sep 30 01:00:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1759999999999999,"15min":1.1799999999999999},"min":{"1min":0.84999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.492,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.1850000000000001,"15min":1.181},"min":{"1min":1.0169999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.3049999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.177}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5756,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366084,"statfs":{"total":96636764160,"available":96630870016,"internally_reserved":0,"allocated":512000,"data_stored":204711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1,"15min":1.046},"min":{"1min":0.70299999999999996,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3120000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0109999999999999,"15min":1.0369999999999999},"min":{"1min":0.71599999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.145,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.115}]},{"osd":1,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.95899999999999996,"15min":0.93500000000000005},"min":{"1min":0.75,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.23,"5min":1.331,"15min":1.331},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.90300000000000002,"5min":17.184999999999999,"15min":6.3789999999999996},"min":{"1min":0.64800000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.399,"5min":897.03999999999996,"15min":897.03999999999996},"last":0.77000000000000002}]},{"osd":2,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":9.843,"15min":33.423000000000002},"min":{"1min":0.81999999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.377,"5min":883.96900000000005,"15min":883.96900000000005},"last":689.13},{"interface":"front","average":{"1min":1.143,"5min":1.1000000000000001,"15min":1.119},"min":{"1min":0.88800000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.329,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.88700000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":40.817999999999998,"5min":9.0139999999999993,"15min":3.734},"min":{"1min":0.84199999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":795.77099999999996,"5min":795.77099999999996,"15min":795.77099999999996},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.0600000000000001,"5min":2.1320000000000001,"15min":1.4470000000000001},"min":{"1min":0.85299999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.3640000000000001,"5min":100.221,"15min":100.221},"last":1.1839999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":3.6760000000000002,"15min":1.8700000000000001},"min":{"1min":0.69299999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2330000000000001,"5min":272.95100000000002,"15min":272.95100000000002},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.0720000000000001,"15min":1.1419999999999999},"min":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.508,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.018}]},{"osd":6,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":3.9969999999999999,"15min":2.0539999999999998},"min":{"1min":0.75800000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3069999999999999,"5min":309.56200000000001,"15min":309.56200000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.194,"5min":3.2570000000000001,"15min":1.869},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.3540000000000001,"5min":216.988,"15min":216.988},"last":1.0940000000000001}]},{"osd":7,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":7.6900000000000004,"15min":3.375},"min":{"1min":0.70199999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.371,"5min":609.06899999999996,"15min":609.06899999999996},"last":1.2270000000000001},{"interface":"front","average":{"1min":28.812000000000001,"5min":11.192,"15min":19.391999999999999},"min":{"1min":0.91400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":553.25900000000001,"5min":553.25900000000001,"15min":553.25900000000001},"last":1.1519999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5692,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366148,"statfs":{"total":96636764160,"available":96630935552,"internally_reserved":0,"allocated":512000,"data_stored":204711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":5.6349999999999998,"15min":17.957999999999998},"min":{"1min":0.77300000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.405,"5min":418.61799999999999,"15min":418.61799999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":47.545000000000002,"5min":10.387,"15min":4.1529999999999996},"min":{"1min":0.83899999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":929.23299999999995,"5min":929.23299999999995,"15min":929.23299999999995},"last":1.0660000000000001}]},{"osd":1,"last update":"Sat Sep 30 01:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.01,"15min":1.02},"min":{"1min":0.77500000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":21.739999999999998,"5min":5.2729999999999997,"15min":2.5230000000000001},"min":{"1min":0.91700000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":329.779,"5min":329.779,"15min":329.779},"last":0.84499999999999997}]},{"osd":2,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0029999999999999,"15min":0.94299999999999995},"min":{"1min":0.68600000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.6619999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.1519999999999999,"5min":10.81,"15min":10.281000000000001},"min":{"1min":0.85899999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.637,"5min":633.17700000000002,"15min":633.17700000000002},"last":966.21100000000001}]},{"osd":3,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":17.687999999999999,"15min":40.594000000000001},"min":{"1min":0.751,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3460000000000001,"5min":918.53399999999999,"15min":918.53399999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":0.997},"min":{"1min":0.73999999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.159,"15min":1.1259999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2989999999999999,"5min":16.021999999999998,"15min":16.021999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.111,"5min":1.0329999999999999,"15min":1.0349999999999999},"min":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.1259999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":6.0229999999999997,"15min":13.205},"min":{"1min":0.72999999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.196,"5min":299.57100000000003,"15min":299.57100000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.224,"15min":1.216},"min":{"1min":1.0169999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6419999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.0209999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.8879999999999999,"15min":1.413},"min":{"1min":0.79900000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.3979999999999999,"5min":88.852000000000004,"15min":88.852000000000004},"last":0.76600000000000001},{"interface":"front","average":{"1min":36.619999999999997,"5min":8.2650000000000006,"15min":3.5550000000000002},"min":{"1min":0.82799999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":501.91399999999999,"5min":501.91399999999999,"15min":501.91399999999999},"last":0.94799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5756,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366084,"statfs":{"total":96636764160,"available":96630870016,"internally_reserved":0,"allocated":512000,"data_stored":204711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.748,"15min":0.72199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1759999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.501},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.5,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0129999999999999,"5min":1.073,"15min":1.073},"last":0.65700000000000003}]},{"osd":1,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":10.505000000000001,"15min":4.0309999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1419999999999999,"5min":873.54200000000003,"15min":873.54200000000003},"last":0.746},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.86099999999999999,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4099999999999999,"5min":1.607,"15min":1.607},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":28.664000000000001,"5min":6.444,"15min":2.7469999999999999},"min":{"1min":0.69999999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":501.649,"5min":501.649,"15min":501.649},"last":0.81000000000000005},{"interface":"front","average":{"1min":74.411000000000001,"5min":15.616,"15min":5.8040000000000003},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":501.74000000000001,"5min":501.74000000000001,"15min":501.74000000000001},"last":0.55800000000000005}]},{"osd":4,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":15.093999999999999,"15min":5.75},"min":{"1min":0.79900000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.292,"5min":735.82799999999997,"15min":735.82799999999997},"last":0.874},{"interface":"front","average":{"1min":1.056,"5min":1.073,"15min":1.1379999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.242,"5min":1.875,"15min":1.875},"last":0.89500000000000002}]},{"osd":5,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":4.9539999999999997,"15min":2.4470000000000001},"min":{"1min":0.90200000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.4370000000000001,"5min":363.78300000000002,"15min":363.78300000000002},"last":1.02},{"interface":"front","average":{"1min":1.161,"5min":1.1299999999999999,"15min":1.163},"min":{"1min":0.95199999999999996,"5min":0.876,"15min":0.876},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.038}]},{"osd":6,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1379999999999999,"15min":1.179},"min":{"1min":0.93000000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.4430000000000001,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":6.0330000000000004,"15min":2.8220000000000001},"min":{"1min":0.95999999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.3500000000000001,"5min":462.90600000000001,"15min":462.90600000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":18.417999999999999,"15min":6.9279999999999999},"min":{"1min":1.0529999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.429,"5min":553.26900000000001,"15min":553.26900000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":42.011000000000003,"5min":21.047999999999998,"15min":28.282},"min":{"1min":0.95699999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":694.64599999999996,"5min":694.64599999999996,"15min":694.64599999999996},"last":0.96999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5752,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366088,"statfs":{"total":96636764160,"available":96630874112,"internally_reserved":0,"allocated":507904,"data_stored":203755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":3.1920000000000002,"15min":1.5609999999999999},"min":{"1min":0.53700000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.504,"5min":272.59399999999999,"15min":272.59399999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":6.5250000000000004,"15min":12.198},"min":{"1min":0.60199999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.206,"5min":298.29399999999998,"15min":298.29399999999998},"last":0.69899999999999995}]},{"osd":1,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":2.746,"15min":1.4379999999999999},"min":{"1min":0.54600000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.294,"5min":217.208,"15min":217.208},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":11.891,"15min":4.5350000000000001},"min":{"1min":0.68999999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2989999999999999,"5min":771.41800000000001,"15min":771.41800000000001},"last":0.66300000000000003}]},{"osd":3,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89100000000000001,"15min":0.93200000000000005},"min":{"1min":0.69099999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.028,"5min":1.782,"15min":1.782},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":5.6840000000000002,"15min":2.5640000000000001},"min":{"1min":0.72899999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8759999999999999,"5min":519.80100000000004,"15min":519.80100000000004},"last":0.628}]},{"osd":4,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.0609999999999999,"15min":1.087},"min":{"1min":0.78600000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.1419999999999999},{"interface":"front","average":{"1min":34.162999999999997,"5min":8.1120000000000001,"15min":3.4460000000000002},"min":{"1min":0.80100000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":501.62200000000001,"5min":501.62200000000001,"15min":501.62200000000001},"last":0.872}]},{"osd":5,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0569999999999999,"15min":1.079},"min":{"1min":0.752,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.3169999999999999,"5min":1.895,"15min":1.895},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.123,"5min":4.1070000000000002,"15min":2.1309999999999998},"min":{"1min":0.89400000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.9930000000000001,"5min":359.505,"15min":359.505},"last":0.93600000000000005}]},{"osd":6,"last update":"Sat Sep 30 01:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":8.6329999999999991,"15min":3.6819999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.468,"5min":786.60400000000004,"15min":786.60400000000004},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.206,"15min":1.2130000000000001},"min":{"1min":0.96599999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.5880000000000001,"5min":1.944,"15min":1.944},"last":1.01}]},{"osd":7,"last update":"Sat Sep 30 01:00:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2210000000000001,"15min":1.238},"min":{"1min":0.88,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0310000000000001,"5min":2.048,"15min":2.048},"last":1.0620000000000001},{"interface":"front","average":{"1min":7.0119999999999996,"5min":2.3420000000000001,"15min":1.5940000000000001},"min":{"1min":0.98299999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":111.39400000000001,"5min":111.39400000000001,"15min":111.39400000000001},"last":0.86399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5756,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366084,"statfs":{"total":96636764160,"available":96630870016,"internally_reserved":0,"allocated":512000,"data_stored":204711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":9.9900000000000002,"15min":16.190000000000001},"min":{"1min":0.53700000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.321,"5min":553.65599999999995,"15min":553.65599999999995},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":6.5229999999999997,"15min":2.7650000000000001},"min":{"1min":0.65800000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.087,"5min":501.34699999999998,"15min":501.34699999999998},"last":0.73199999999999998}]},{"osd":2,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":4.0469999999999997,"15min":1.865},"min":{"1min":0.57499999999999996,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91300000000000003,"5min":313.49000000000001,"15min":313.49000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.86099999999999999,"15min":0.84299999999999997},"min":{"1min":0.60499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.282,"5min":1.319,"15min":1.319},"last":0.499}]},{"osd":3,"last update":"Sat Sep 30 01:01:01 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.92400000000000004,"15min":0.91200000000000003},"min":{"1min":0.747,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.60399999999999998},{"interface":"front","average":{"1min":55.460999999999999,"5min":31.306000000000001,"15min":11.065},"min":{"1min":0.78300000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":873.47900000000004,"5min":873.47900000000004,"15min":873.47900000000004},"last":0.88200000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.042,"15min":1.0880000000000001},"min":{"1min":0.76100000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.502,"5min":1.556,"15min":1.556},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1160000000000001,"15min":1.1180000000000001},"min":{"1min":0.88700000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.79600000000000004}]},{"osd":5,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0529999999999999,"15min":1.089},"min":{"1min":0.83099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5169999999999999,"5min":1.71,"15min":1.71},"last":1.0249999999999999},{"interface":"front","average":{"1min":37.758000000000003,"5min":10.032999999999999,"15min":6.3899999999999997},"min":{"1min":0.85499999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":586.91999999999996,"5min":586.91999999999996,"15min":586.91999999999996},"last":1.0669999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.123,"15min":1.161},"min":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.464,"5min":1.77,"15min":1.77},"last":1.548},{"interface":"front","average":{"1min":1.2110000000000001,"5min":6.8920000000000003,"15min":3.0840000000000001},"min":{"1min":1.0920000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.306,"5min":544.41600000000005,"15min":544.41600000000005},"last":1.214}]},{"osd":7,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.173,"15min":1.163},"min":{"1min":1.016,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.49,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.234,"5min":1.21,"15min":1.242},"min":{"1min":1.0549999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4279999999999999,"5min":1.861,"15min":1.861},"last":1.2410000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6264,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365576,"statfs":{"total":96636764160,"available":96630349824,"internally_reserved":0,"allocated":573440,"data_stored":733639,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":7.1059999999999999,"15min":9.6199999999999992},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.99299999999999999,"5min":309.20299999999997,"15min":309.20299999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":5.4189999999999996,"15min":2.3159999999999998},"min":{"1min":0.60799999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98099999999999998,"5min":533.17499999999995,"15min":533.17499999999995},"last":0.76600000000000001}]},{"osd":2,"last update":"Sat Sep 30 01:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":7.6079999999999997,"15min":25.683},"min":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.044,"5min":676.57399999999996,"15min":676.57399999999996},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":18.818999999999999,"15min":6.8559999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.123,"5min":898.26700000000005,"15min":898.26700000000005},"last":0.88100000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":4.0250000000000004,"15min":1.897},"min":{"1min":0.70599999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.214,"5min":362.38999999999999,"15min":362.38999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":1.0109999999999999},"min":{"1min":0.73499999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3360000000000001,"5min":1.585,"15min":1.585},"last":1.1040000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.05,"15min":1.0569999999999999},"min":{"1min":0.90700000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.1919999999999999,"5min":1.377,"15min":1.377},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.117,"15min":1.1539999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.21,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.216}]},{"osd":5,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":12.81,"15min":5.0179999999999998},"min":{"1min":0.79600000000000004,"5min":0.75,"15min":0.75},"max":{"1min":1.4059999999999999,"5min":667.67700000000002,"15min":667.67700000000002},"last":1.304},{"interface":"front","average":{"1min":1.107,"5min":4.2110000000000003,"15min":2.1120000000000001},"min":{"1min":0.97599999999999998,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.367,"5min":360.90100000000001,"15min":360.90100000000001},"last":1.0109999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":8.8379999999999992,"5min":2.6869999999999998,"15min":1.72},"min":{"1min":0.88900000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":163.465,"5min":163.465,"15min":163.465},"last":1.143},{"interface":"front","average":{"1min":1.05,"5min":14.26,"15min":5.5599999999999996},"min":{"1min":0.877,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.3540000000000001,"5min":970.15200000000004,"15min":970.15200000000004},"last":1.0369999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:01:05 2023","interfaces":[{"interface":"back","average":{"1min":45.124000000000002,"5min":15.372999999999999,"15min":10.071},"min":{"1min":0.94899999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":924.71799999999996,"5min":924.71799999999996,"15min":924.71799999999996},"last":1.264},{"interface":"front","average":{"1min":1.1879999999999999,"5min":13.082000000000001,"15min":5.1680000000000001},"min":{"1min":1.01,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.391,"5min":501.84800000000001,"15min":501.84800000000001},"last":1.179}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:01:09.286 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-09-30T01:00:02.106695+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=0, tm_sec=2, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=1, tm_sec=4, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:01:09.286 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-09-30T01:00:00.082925+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=0, tm_sec=0, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=1, tm_sec=4, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:01:09.287 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-09-30T00:59:52.818607+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=0, tm_min=59, tm_sec=52, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=1, tm_sec=4, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:01:09.287 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-09-30T01:00:11.015207+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=0, tm_sec=11, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=1, tm_sec=4, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:01:09.287 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-09-30T00:59:56.922501+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=0, tm_min=59, tm_sec=56, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=1, tm_sec=4, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:01:09.287 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-09-30T00:59:48.922299+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=0, tm_min=59, tm_sec=48, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=1, tm_sec=4, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:01:09.287 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-09-30T00:59:49.817496+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=0, tm_min=59, tm_sec=49, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=1, tm_sec=4, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:01:09.287 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-09-30T00:59:37.952950+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=0, tm_min=59, tm_sec=37, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=1, tm_sec=4, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:01:09.287 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-30T01:00:04.741859+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=0, tm_sec=4, tm_wday=5, tm_yday=273, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=30, tm_hour=1, tm_min=1, tm_sec=4, tm_wday=5, tm_yday=273, tm_isdst=0) 2023-09-30T01:01:09.288 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-30T01:01:29.289 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T01:01:29.596 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-30T01:01:29.598 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-09-30T01:01:29.610 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2023-09-30T01:01:28.497092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":47504,"kb_used_data":4176,"kb_used_omap":0,"kb_used_meta":43264,"kb_avail":754927216,"statfs":{"total":773094113280,"available":773045469184,"internally_reserved":0,"allocated":4276224,"data_stored":3223516,"data_compressed":89871,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":44302336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003589"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:15.590178+0000","last_change":"2023-09-30T01:01:15.590096+0000","last_active":"2023-09-30T01:01:15.590178+0000","last_peered":"2023-09-30T01:01:15.590178+0000","last_clean":"2023-09-30T01:01:15.590178+0000","last_became_active":"2023-09-30T00:55:55.343702+0000","last_became_peered":"2023-09-30T00:55:55.343702+0000","last_unstale":"2023-09-30T01:01:15.590178+0000","last_undegraded":"2023-09-30T01:01:15.590178+0000","last_fullsized":"2023-09-30T01:01:15.590178+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:01:15.589868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:01:15.589868+0000","last_clean_scrub_stamp":"2023-09-30T01:01:15.589868+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:02:25.993748+0000","scrub_duration":0.0040577230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:10.254163+0000","last_change":"2023-09-30T01:01:10.254079+0000","last_active":"2023-09-30T01:01:10.254163+0000","last_peered":"2023-09-30T01:01:10.254163+0000","last_clean":"2023-09-30T01:01:10.254163+0000","last_became_active":"2023-09-30T00:55:55.372455+0000","last_became_peered":"2023-09-30T00:55:55.372455+0000","last_unstale":"2023-09-30T01:01:10.254163+0000","last_undegraded":"2023-09-30T01:01:10.254163+0000","last_fullsized":"2023-09-30T01:01:10.254163+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:01:10.253811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:01:10.253811+0000","last_clean_scrub_stamp":"2023-09-30T01:01:10.253811+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:02:23.816275+0000","scrub_duration":0.004498976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:15.403271+0000","last_change":"2023-09-30T01:01:15.403184+0000","last_active":"2023-09-30T01:01:15.403271+0000","last_peered":"2023-09-30T01:01:15.403271+0000","last_clean":"2023-09-30T01:01:15.403271+0000","last_became_active":"2023-09-30T00:55:55.371497+0000","last_became_peered":"2023-09-30T00:55:55.371497+0000","last_unstale":"2023-09-30T01:01:15.403271+0000","last_undegraded":"2023-09-30T01:01:15.403271+0000","last_fullsized":"2023-09-30T01:01:15.403271+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:01:15.402948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:01:15.402948+0000","last_clean_scrub_stamp":"2023-09-30T01:01:15.402948+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:02:42.768351+0000","scrub_duration":0.0037972209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:11.207796+0000","last_change":"2023-09-30T01:01:11.207714+0000","last_active":"2023-09-30T01:01:11.207796+0000","last_peered":"2023-09-30T01:01:11.207796+0000","last_clean":"2023-09-30T01:01:11.207796+0000","last_became_active":"2023-09-30T00:55:55.350964+0000","last_became_peered":"2023-09-30T00:55:55.350964+0000","last_unstale":"2023-09-30T01:01:11.207796+0000","last_undegraded":"2023-09-30T01:01:11.207796+0000","last_fullsized":"2023-09-30T01:01:11.207796+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:01:11.207435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:01:11.207435+0000","last_clean_scrub_stamp":"2023-09-30T01:01:11.207435+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:02:20.231743+0000","scrub_duration":0.0040395960000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:11.942723+0000","last_change":"2023-09-30T01:01:11.942633+0000","last_active":"2023-09-30T01:01:11.942723+0000","last_peered":"2023-09-30T01:01:11.942723+0000","last_clean":"2023-09-30T01:01:11.942723+0000","last_became_active":"2023-09-30T00:55:55.342131+0000","last_became_peered":"2023-09-30T00:55:55.342131+0000","last_unstale":"2023-09-30T01:01:11.942723+0000","last_undegraded":"2023-09-30T01:01:11.942723+0000","last_fullsized":"2023-09-30T01:01:11.942723+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:01:11.942406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:01:11.942406+0000","last_clean_scrub_stamp":"2023-09-30T01:01:11.942406+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:02:13.445733+0000","scrub_duration":0.0038800839999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:08.512726+0000","last_change":"2023-09-30T01:01:08.512682+0000","last_active":"2023-09-30T01:01:08.512726+0000","last_peered":"2023-09-30T01:01:08.512726+0000","last_clean":"2023-09-30T01:01:08.512726+0000","last_became_active":"2023-09-30T00:55:55.350971+0000","last_became_peered":"2023-09-30T00:55:55.350971+0000","last_unstale":"2023-09-30T01:01:08.512726+0000","last_undegraded":"2023-09-30T01:01:08.512726+0000","last_fullsized":"2023-09-30T01:01:08.512726+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:01:08.512508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:01:08.512508+0000","last_clean_scrub_stamp":"2023-09-30T01:01:08.512508+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:02:19.044648+0000","scrub_duration":0.0036105999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:16.378912+0000","last_change":"2023-09-30T01:01:16.378812+0000","last_active":"2023-09-30T01:01:16.378912+0000","last_peered":"2023-09-30T01:01:16.378912+0000","last_clean":"2023-09-30T01:01:16.378912+0000","last_became_active":"2023-09-30T00:55:55.350574+0000","last_became_peered":"2023-09-30T00:55:55.350574+0000","last_unstale":"2023-09-30T01:01:16.378912+0000","last_undegraded":"2023-09-30T01:01:16.378912+0000","last_fullsized":"2023-09-30T01:01:16.378912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:01:16.378558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:01:16.378558+0000","last_clean_scrub_stamp":"2023-09-30T01:01:16.378558+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:02:43.050520+0000","scrub_duration":0.004072539,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:08.985195+0000","last_change":"2023-09-30T01:01:08.985060+0000","last_active":"2023-09-30T01:01:08.985195+0000","last_peered":"2023-09-30T01:01:08.985195+0000","last_clean":"2023-09-30T01:01:08.985195+0000","last_became_active":"2023-09-30T00:55:55.351023+0000","last_became_peered":"2023-09-30T00:55:55.351023+0000","last_unstale":"2023-09-30T01:01:08.985195+0000","last_undegraded":"2023-09-30T01:01:08.985195+0000","last_fullsized":"2023-09-30T01:01:08.985195+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T01:01:08.984829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T01:01:08.984829+0000","last_clean_scrub_stamp":"2023-09-30T01:01:08.984829+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:02:29.884400+0000","scrub_duration":0.0038934920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"45'119","reported_seq":231,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-09-30T01:01:11.499564+0000","last_change":"2023-09-30T01:01:11.499513+0000","last_active":"2023-09-30T01:01:11.499564+0000","last_peered":"2023-09-30T01:01:11.499564+0000","last_clean":"2023-09-30T01:01:11.499564+0000","last_became_active":"2023-09-30T00:55:54.335401+0000","last_became_peered":"2023-09-30T00:55:54.335401+0000","last_unstale":"2023-09-30T01:01:11.499564+0000","last_undegraded":"2023-09-30T01:01:11.499564+0000","last_fullsized":"2023-09-30T01:01:11.499564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'119","last_scrub_stamp":"2023-09-30T01:01:11.499380+0000","last_deep_scrub":"45'119","last_deep_scrub_stamp":"2023-09-30T01:01:11.499380+0000","last_clean_scrub_stamp":"2023-09-30T01:01:11.499380+0000","objects_scrubbed":2,"log_size":119,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T01:02:18.736666+0000","scrub_duration":0.011477746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":1586784,"data_compressed":89871,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6264,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365576,"statfs":{"total":96636764160,"available":96630349824,"internally_reserved":0,"allocated":573440,"data_stored":733639,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":1.054,"15min":1.514},"min":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.135,"5min":18.013999999999999,"15min":18.013999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.127,"15min":1.6519999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.145,"5min":20.856999999999999,"15min":20.856999999999999},"last":1.018}]},{"osd":1,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.849,"15min":4.306},"min":{"1min":0.63800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.24,"5min":50.606999999999999,"15min":50.606999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.357,"15min":2.4209999999999998},"min":{"1min":0.70699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.4199999999999999,"5min":41.756999999999998,"15min":41.756999999999998},"last":0.91500000000000004}]},{"osd":2,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.1739999999999999,"15min":1.641},"min":{"1min":0.83699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.1080000000000001,"5min":17.486999999999998,"15min":17.486999999999998},"last":1.1379999999999999},{"interface":"front","average":{"1min":23.733000000000001,"5min":8.5530000000000008,"15min":4.1509999999999998},"min":{"1min":0.86899999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":364.00400000000002,"5min":364.00400000000002,"15min":364.00400000000002},"last":1.101}]},{"osd":3,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":39.890000000000001,"5min":9.0020000000000007,"15min":4.1059999999999999},"min":{"1min":0.626,"5min":0.621,"15min":0.621},"max":{"1min":501.76100000000002,"5min":501.76100000000002,"15min":501.76100000000002},"last":1.054},{"interface":"front","average":{"1min":30.462,"5min":7.0800000000000001,"15min":3.5630000000000002},"min":{"1min":0.80800000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":501.42500000000001,"5min":501.42500000000001,"15min":501.42500000000001},"last":1.181}]},{"osd":4,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.03,"15min":1.1819999999999999},"min":{"1min":0.70399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.482,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":15.436,"15min":6.0030000000000001},"min":{"1min":0.748,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2170000000000001,"5min":829.03899999999999,"15min":829.03899999999999},"last":1.22}]},{"osd":5,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":8.3789999999999996,"15min":3.7149999999999999},"min":{"1min":0.84899999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.393,"5min":830.25300000000004,"15min":830.25300000000004},"last":1.266},{"interface":"front","average":{"1min":1.0469999999999999,"5min":6.4749999999999996,"15min":3.1070000000000002},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.2330000000000001,"5min":530.12199999999996,"15min":530.12199999999996},"last":1.2869999999999999}]},{"osd":6,"last update":"Sat Sep 30 01:00:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":9.6679999999999993,"15min":4.21},"min":{"1min":0.86699999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.3080000000000001,"5min":971.51400000000001,"15min":971.51400000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.2110000000000001,"15min":1.359},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.2370000000000001,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.3120000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6264,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365576,"statfs":{"total":96636764160,"available":96630349824,"internally_reserved":0,"allocated":573440,"data_stored":733639,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":7.2350000000000003,"15min":3.0720000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.2689999999999999,"5min":500.60300000000001,"15min":500.60300000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94699999999999995,"15min":0.94199999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.244,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.1000000000000001}]},{"osd":1,"last update":"Sat Sep 30 01:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":10.351000000000001,"15min":4.0570000000000004},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.266,"5min":688.31500000000005,"15min":688.31500000000005},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":5.7400000000000002,"15min":2.6030000000000002},"min":{"1min":0.69799999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.141,"5min":501.56200000000001,"15min":501.56200000000001},"last":0.84599999999999997}]},{"osd":2,"last update":"Sat Sep 30 01:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":2.9140000000000001,"15min":1.641},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.377,"5min":201.053,"15min":201.053},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.004,"15min":0.98799999999999999},"min":{"1min":0.79900000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.333,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Sat Sep 30 01:01:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":8.2810000000000006,"15min":3.4790000000000001},"min":{"1min":0.76900000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.304,"5min":756.92499999999995,"15min":756.92499999999995},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.012,"5min":21.189,"15min":7.7629999999999999},"min":{"1min":0.89400000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.159,"5min":859.12800000000004,"15min":859.12800000000004},"last":1.0580000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:01:06 2023","interfaces":[{"interface":"back","average":{"1min":31.451000000000001,"5min":13.433999999999999,"15min":5.1470000000000002},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":580.18399999999997,"5min":580.18399999999997,"15min":580.18399999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":38.704999999999998,"5min":13.611000000000001,"15min":5.2240000000000002},"min":{"1min":0.71699999999999997,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":716.02200000000005,"5min":716.02200000000005,"15min":716.02200000000005},"last":0.77600000000000002}]},{"osd":5,"last update":"Sat Sep 30 01:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":8.7390000000000008,"15min":3.5939999999999999},"min":{"1min":0.82999999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4410000000000001,"5min":651.43100000000004,"15min":651.43100000000004},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.079,"15min":1.083},"min":{"1min":0.82299999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.2190000000000001,"5min":1.641,"15min":1.641},"last":0.97899999999999998}]},{"osd":7,"last update":"Sat Sep 30 01:01:06 2023","interfaces":[{"interface":"back","average":{"1min":43.152000000000001,"5min":9.5700000000000003,"15min":3.9780000000000002},"min":{"1min":0.93700000000000006,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":798.25999999999999,"5min":798.25999999999999,"15min":798.25999999999999},"last":1.034},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1799999999999999,"15min":1.179},"min":{"1min":0.97599999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.351,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.1879999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5756,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366084,"statfs":{"total":96636764160,"available":96630870016,"internally_reserved":0,"allocated":512000,"data_stored":204711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1,"15min":1.046},"min":{"1min":0.70299999999999996,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3120000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0109999999999999,"15min":1.0369999999999999},"min":{"1min":0.71599999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.145,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.0329999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.95899999999999996,"15min":0.93500000000000005},"min":{"1min":0.75,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.23,"5min":1.331,"15min":1.331},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":17.184999999999999,"15min":6.3789999999999996},"min":{"1min":0.64800000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.399,"5min":897.03999999999996,"15min":897.03999999999996},"last":0.82599999999999996}]},{"osd":2,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":9.843,"15min":33.423000000000002},"min":{"1min":0.81999999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.377,"5min":883.96900000000005,"15min":883.96900000000005},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.1000000000000001,"15min":1.119},"min":{"1min":0.88800000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.329,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.248}]},{"osd":3,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":40.817999999999998,"5min":9.0139999999999993,"15min":3.734},"min":{"1min":0.84199999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":795.77099999999996,"5min":795.77099999999996,"15min":795.77099999999996},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":2.1320000000000001,"15min":1.4470000000000001},"min":{"1min":0.85299999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.3640000000000001,"5min":100.221,"15min":100.221},"last":1.147}]},{"osd":4,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":3.6760000000000002,"15min":1.8700000000000001},"min":{"1min":0.69299999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2330000000000001,"5min":272.95100000000002,"15min":272.95100000000002},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.0720000000000001,"15min":1.1419999999999999},"min":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.508,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.95399999999999996}]},{"osd":6,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":3.9969999999999999,"15min":2.0539999999999998},"min":{"1min":0.75800000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3069999999999999,"5min":309.56200000000001,"15min":309.56200000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.194,"5min":3.2570000000000001,"15min":1.869},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.3540000000000001,"5min":216.988,"15min":216.988},"last":1.012}]},{"osd":7,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":7.6900000000000004,"15min":3.375},"min":{"1min":0.70199999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.371,"5min":609.06899999999996,"15min":609.06899999999996},"last":0.91200000000000003},{"interface":"front","average":{"1min":28.812000000000001,"5min":11.192,"15min":19.391999999999999},"min":{"1min":0.91400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":553.25900000000001,"5min":553.25900000000001,"15min":553.25900000000001},"last":1.2729999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574918,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5692,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366148,"statfs":{"total":96636764160,"available":96630935552,"internally_reserved":0,"allocated":512000,"data_stored":204711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":5.6349999999999998,"15min":17.957999999999998},"min":{"1min":0.77300000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.405,"5min":418.61799999999999,"15min":418.61799999999999},"last":0.875},{"interface":"front","average":{"1min":47.545000000000002,"5min":10.387,"15min":4.1529999999999996},"min":{"1min":0.83899999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":929.23299999999995,"5min":929.23299999999995,"15min":929.23299999999995},"last":1.2110000000000001}]},{"osd":1,"last update":"Sat Sep 30 01:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.01,"15min":1.02},"min":{"1min":0.77500000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.02},{"interface":"front","average":{"1min":21.739999999999998,"5min":5.2729999999999997,"15min":2.5230000000000001},"min":{"1min":0.91700000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":329.779,"5min":329.779,"15min":329.779},"last":1.2350000000000001}]},{"osd":2,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0029999999999999,"15min":0.94299999999999995},"min":{"1min":0.68600000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.6619999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.1519999999999999,"5min":10.81,"15min":10.281000000000001},"min":{"1min":0.85899999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.637,"5min":633.17700000000002,"15min":633.17700000000002},"last":1.1830000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":17.687999999999999,"15min":40.594000000000001},"min":{"1min":0.751,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3460000000000001,"5min":918.53399999999999,"15min":918.53399999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":0.997},"min":{"1min":0.73999999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.159,"15min":1.1259999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2989999999999999,"5min":16.021999999999998,"15min":16.021999999999998},"last":0.751},{"interface":"front","average":{"1min":1.111,"5min":1.0329999999999999,"15min":1.0349999999999999},"min":{"1min":0.70399999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.97199999999999998}]},{"osd":6,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":6.0229999999999997,"15min":13.205},"min":{"1min":0.72999999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.196,"5min":299.57100000000003,"15min":299.57100000000003},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.224,"15min":1.216},"min":{"1min":1.0169999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6419999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.1339999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:01:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.8879999999999999,"15min":1.413},"min":{"1min":0.79900000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.3979999999999999,"5min":88.852000000000004,"15min":88.852000000000004},"last":1.048},{"interface":"front","average":{"1min":36.619999999999997,"5min":8.2650000000000006,"15min":3.5550000000000002},"min":{"1min":0.82799999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":501.91399999999999,"5min":501.91399999999999,"15min":501.91399999999999},"last":1.085}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5756,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366084,"statfs":{"total":96636764160,"available":96630870016,"internally_reserved":0,"allocated":512000,"data_stored":204711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.748,"15min":0.72199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1759999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.5,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0129999999999999,"5min":1.073,"15min":1.073},"last":0.72999999999999998}]},{"osd":1,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":10.505000000000001,"15min":4.0309999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1419999999999999,"5min":873.54200000000003,"15min":873.54200000000003},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.86099999999999999,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4099999999999999,"5min":1.607,"15min":1.607},"last":0.79600000000000004}]},{"osd":2,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":28.664000000000001,"5min":6.444,"15min":2.7469999999999999},"min":{"1min":0.69999999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":501.649,"5min":501.649,"15min":501.649},"last":0.84099999999999997},{"interface":"front","average":{"1min":74.411000000000001,"5min":15.616,"15min":5.8040000000000003},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":501.74000000000001,"5min":501.74000000000001,"15min":501.74000000000001},"last":1.0029999999999999}]},{"osd":4,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":15.093999999999999,"15min":5.75},"min":{"1min":0.79900000000000004,"5min":0.748,"15min":0.748},"max":{"1min":1.292,"5min":735.82799999999997,"15min":735.82799999999997},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.073,"15min":1.1379999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.242,"5min":1.875,"15min":1.875},"last":0.88700000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":4.9539999999999997,"15min":2.4470000000000001},"min":{"1min":0.90200000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.4370000000000001,"5min":363.78300000000002,"15min":363.78300000000002},"last":1.046},{"interface":"front","average":{"1min":1.161,"5min":1.1299999999999999,"15min":1.163},"min":{"1min":0.95199999999999996,"5min":0.876,"15min":0.876},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.1120000000000001}]},{"osd":6,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1379999999999999,"15min":1.179},"min":{"1min":0.93000000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.4430000000000001,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.135},{"interface":"front","average":{"1min":1.1850000000000001,"5min":6.0330000000000004,"15min":2.8220000000000001},"min":{"1min":0.95999999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.3500000000000001,"5min":462.90600000000001,"15min":462.90600000000001},"last":1.1579999999999999}]},{"osd":7,"last update":"Sat Sep 30 01:01:03 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":18.417999999999999,"15min":6.9279999999999999},"min":{"1min":1.0529999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.429,"5min":553.26900000000001,"15min":553.26900000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":42.011000000000003,"5min":21.047999999999998,"15min":28.282},"min":{"1min":0.95699999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":694.64599999999996,"5min":694.64599999999996,"15min":694.64599999999996},"last":1.2030000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5752,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366088,"statfs":{"total":96636764160,"available":96630874112,"internally_reserved":0,"allocated":507904,"data_stored":203755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":3.1840000000000002,"15min":1.5580000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1180000000000001,"5min":272.59399999999999,"15min":272.59399999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":27.605,"5min":9.0389999999999997,"15min":13.036},"min":{"1min":0.59899999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":510.05200000000002,"5min":510.05200000000002,"15min":510.05200000000002},"last":1.2769999999999999}]},{"osd":1,"last update":"Sat Sep 30 01:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":2.7519999999999998,"15min":1.4399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.655,"5min":217.208,"15min":217.208},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":11.881,"15min":4.532},"min":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1459999999999999,"5min":771.41800000000001,"15min":771.41800000000001},"last":1.1719999999999999}]},{"osd":3,"last update":"Sat Sep 30 01:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.86899999999999999,"15min":0.92500000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1599999999999999,"5min":1.782,"15min":1.782},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.94699999999999995,"5min":5.6719999999999997,"15min":2.5600000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.5209999999999999,"5min":519.80100000000004,"15min":519.80100000000004},"last":1.1970000000000001}]},{"osd":4,"last update":"Sat Sep 30 01:01:09 2023","interfaces":[{"interface":"back","average":{"1min":30.315999999999999,"5min":6.9039999999999999,"15min":3.0339999999999998},"min":{"1min":0.80500000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":585.779,"5min":585.779,"15min":585.779},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.05,"5min":8.0990000000000002,"15min":3.4420000000000002},"min":{"1min":0.80000000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.651,"5min":501.62200000000001,"15min":501.62200000000001},"last":1.23}]},{"osd":5,"last update":"Sat Sep 30 01:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0509999999999999,"15min":1.077},"min":{"1min":0.78800000000000003,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.393,"5min":1.895,"15min":1.895},"last":1.091},{"interface":"front","average":{"1min":50.384999999999998,"5min":13.955,"15min":5.4130000000000003},"min":{"1min":0.90000000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":937.02300000000002,"5min":937.02300000000002,"15min":937.02300000000002},"last":1.1180000000000001}]},{"osd":6,"last update":"Sat Sep 30 01:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":8.6329999999999991,"15min":3.6819999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.468,"5min":786.60400000000004,"15min":786.60400000000004},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.206,"15min":1.2130000000000001},"min":{"1min":0.96599999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.5880000000000001,"5min":1.944,"15min":1.944},"last":1.079}]},{"osd":7,"last update":"Sat Sep 30 01:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2030000000000001,"15min":1.232},"min":{"1min":0.89600000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.6060000000000001,"5min":2.048,"15min":2.048},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.216,"5min":2.3410000000000002,"15min":1.5940000000000001},"min":{"1min":0.80800000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.5640000000000001,"5min":111.39400000000001,"15min":111.39400000000001},"last":1.2509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5756,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94366084,"statfs":{"total":96636764160,"available":96630870016,"internally_reserved":0,"allocated":512000,"data_stored":204711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":9.9900000000000002,"15min":16.190000000000001},"min":{"1min":0.53700000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.321,"5min":553.65599999999995,"15min":553.65599999999995},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":6.5229999999999997,"15min":2.7650000000000001},"min":{"1min":0.65800000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.087,"5min":501.34699999999998,"15min":501.34699999999998},"last":0.70499999999999996}]},{"osd":2,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":4.0469999999999997,"15min":1.865},"min":{"1min":0.57499999999999996,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91300000000000003,"5min":313.49000000000001,"15min":313.49000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.86099999999999999,"15min":0.84299999999999997},"min":{"1min":0.60499999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.282,"5min":1.319,"15min":1.319},"last":1.0389999999999999}]},{"osd":3,"last update":"Sat Sep 30 01:01:01 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.92400000000000004,"15min":0.91200000000000003},"min":{"1min":0.747,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.90400000000000003},{"interface":"front","average":{"1min":55.460999999999999,"5min":31.306000000000001,"15min":11.065},"min":{"1min":0.78300000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":873.47900000000004,"5min":873.47900000000004,"15min":873.47900000000004},"last":0.79800000000000004}]},{"osd":4,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.042,"15min":1.0880000000000001},"min":{"1min":0.76100000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.502,"5min":1.556,"15min":1.556},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1160000000000001,"15min":1.1180000000000001},"min":{"1min":0.88700000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.1970000000000001}]},{"osd":5,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0529999999999999,"15min":1.089},"min":{"1min":0.83099999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5169999999999999,"5min":1.71,"15min":1.71},"last":1.0580000000000001},{"interface":"front","average":{"1min":37.758000000000003,"5min":10.032999999999999,"15min":6.3899999999999997},"min":{"1min":0.85499999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":586.91999999999996,"5min":586.91999999999996,"15min":586.91999999999996},"last":1.089}]},{"osd":6,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.123,"15min":1.161},"min":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.464,"5min":1.77,"15min":1.77},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.2110000000000001,"5min":6.8920000000000003,"15min":3.0840000000000001},"min":{"1min":1.0920000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.306,"5min":544.41600000000005,"15min":544.41600000000005},"last":1.163}]},{"osd":7,"last update":"Sat Sep 30 01:01:00 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.173,"15min":1.163},"min":{"1min":1.016,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.49,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.145},{"interface":"front","average":{"1min":1.234,"5min":1.21,"15min":1.242},"min":{"1min":1.0549999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4279999999999999,"5min":1.861,"15min":1.861},"last":0.97199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6264,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94365576,"statfs":{"total":96636764160,"available":96630349824,"internally_reserved":0,"allocated":573440,"data_stored":733639,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":7.1059999999999999,"15min":9.6199999999999992},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.99299999999999999,"5min":309.20299999999997,"15min":309.20299999999997},"last":1.042},{"interface":"front","average":{"1min":0.76100000000000001,"5min":5.4189999999999996,"15min":2.3159999999999998},"min":{"1min":0.60799999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98099999999999998,"5min":533.17499999999995,"15min":533.17499999999995},"last":0.78600000000000003}]},{"osd":2,"last update":"Sat Sep 30 01:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":7.6079999999999997,"15min":25.683},"min":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.044,"5min":676.57399999999996,"15min":676.57399999999996},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":18.818999999999999,"15min":6.8559999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.123,"5min":898.26700000000005,"15min":898.26700000000005},"last":0.88800000000000001}]},{"osd":3,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":4.0250000000000004,"15min":1.897},"min":{"1min":0.70599999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.214,"5min":362.38999999999999,"15min":362.38999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":1.0109999999999999},"min":{"1min":0.73499999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3360000000000001,"5min":1.585,"15min":1.585},"last":0.52600000000000002}]},{"osd":4,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.05,"15min":1.0569999999999999},"min":{"1min":0.90700000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.1919999999999999,"5min":1.377,"15min":1.377},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.117,"15min":1.1539999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.21,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.98999999999999999}]},{"osd":5,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":12.81,"15min":5.0179999999999998},"min":{"1min":0.79600000000000004,"5min":0.75,"15min":0.75},"max":{"1min":1.4059999999999999,"5min":667.67700000000002,"15min":667.67700000000002},"last":0.745},{"interface":"front","average":{"1min":1.107,"5min":4.2110000000000003,"15min":2.1120000000000001},"min":{"1min":0.97599999999999998,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.367,"5min":360.90100000000001,"15min":360.90100000000001},"last":1.0880000000000001}]},{"osd":6,"last update":"Sat Sep 30 01:00:56 2023","interfaces":[{"interface":"back","average":{"1min":8.8379999999999992,"5min":2.6869999999999998,"15min":1.72},"min":{"1min":0.88900000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":163.465,"5min":163.465,"15min":163.465},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.05,"5min":14.26,"15min":5.5599999999999996},"min":{"1min":0.877,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.3540000000000001,"5min":970.15200000000004,"15min":970.15200000000004},"last":1.3560000000000001}]},{"osd":7,"last update":"Sat Sep 30 01:01:05 2023","interfaces":[{"interface":"back","average":{"1min":45.124000000000002,"5min":15.372999999999999,"15min":10.071},"min":{"1min":0.94899999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":924.71799999999996,"5min":924.71799999999996,"15min":924.71799999999996},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":13.082000000000001,"15min":5.1680000000000001},"min":{"1min":1.01,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.391,"5min":501.84800000000001,"15min":501.84800000000001},"last":1.123}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29957,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T01:01:29.613 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-09-30T01:01:29.945 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1895, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1472, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph_manager.py", line 3189, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-09-30T01:01:29.947 INFO:teuthology.misc:Shutting down mds daemons... 2023-09-30T01:01:29.947 INFO:teuthology.misc:Shutting down osd daemons... 2023-09-30T01:01:29.948 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-30T01:01:29.948 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.009 INFO:tasks.ceph.osd.0:Stopped 2023-09-30T01:01:30.010 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-30T01:01:30.010 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.077 INFO:tasks.ceph.osd.1:Stopped 2023-09-30T01:01:30.078 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-30T01:01:30.078 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.160 INFO:tasks.ceph.osd.2:Stopped 2023-09-30T01:01:30.160 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-30T01:01:30.160 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.237 INFO:tasks.ceph.osd.3:Stopped 2023-09-30T01:01:30.237 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-09-30T01:01:30.238 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.304 INFO:tasks.ceph.osd.4:Stopped 2023-09-30T01:01:30.304 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-09-30T01:01:30.305 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.381 INFO:tasks.ceph.osd.5:Stopped 2023-09-30T01:01:30.381 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-09-30T01:01:30.381 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.445 INFO:tasks.ceph.osd.6:Stopped 2023-09-30T01:01:30.445 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-09-30T01:01:30.445 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.538 INFO:tasks.ceph.osd.7:Stopped 2023-09-30T01:01:30.538 INFO:teuthology.misc:Shutting down mgr daemons... 2023-09-30T01:01:30.538 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-09-30T01:01:30.538 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.564 INFO:tasks.ceph.mgr.y:Stopped 2023-09-30T01:01:30.565 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-30T01:01:30.565 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.597 INFO:tasks.ceph.mgr.x:Stopped 2023-09-30T01:01:30.597 INFO:teuthology.misc:Shutting down mon daemons... 2023-09-30T01:01:30.598 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-30T01:01:30.598 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.619 INFO:tasks.ceph.mon.a:Stopped 2023-09-30T01:01:30.619 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-30T01:01:30.619 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.634 INFO:tasks.ceph.mon.c:Stopped 2023-09-30T01:01:30.635 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-30T01:01:30.635 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T01:01:30.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-09-30T01:01:30.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-09-30T01:01:30.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-09-30T01:01:30.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-09-30T01:01:30.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-09-30T01:01:30.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-09-30T01:01:30.655 INFO:tasks.ceph.mon.b:Stopped 2023-09-30T01:01:30.655 INFO:tasks.ceph:Checking cluster log for badness... 2023-09-30T01:01:30.656 DEBUG:teuthology.orchestra.run.smithi069:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'must scrub before tier agent can activate' | head -n 1 2023-09-30T01:01:30.675 INFO:teuthology.orchestra.run.smithi069.stdout:1696035346.422124 mgr.y (mgr.4101) 1 : cluster [ERR] Unhandled exception from module 'pg_autoscaler' while running on mgr.y: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-30T01:01:30.676 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-09-30T01:01:30.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'must scrub before tier agent can activate' | head -n 1 2023-09-30T01:01:30.738 DEBUG:teuthology.orchestra.run.smithi069:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'must scrub before tier agent can activate' | head -n 1 2023-09-30T01:01:30.753 INFO:teuthology.orchestra.run.smithi069.stdout:1696035346.422124 mgr.y (mgr.4101) 1 : cluster [ERR] Unhandled exception from module 'pg_autoscaler' while running on mgr.y: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-30T01:01:30.753 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi069.front.sepia.ceph.com 2023-09-30T01:01:30.753 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-09-30T01:01:31.065 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi069.front.sepia.ceph.com 2023-09-30T01:01:31.065 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-09-30T01:01:31.242 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi069.front.sepia.ceph.com 2023-09-30T01:01:31.242 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-09-30T01:01:31.417 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi069.front.sepia.ceph.com 2023-09-30T01:01:31.417 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-09-30T01:01:31.614 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi150.front.sepia.ceph.com 2023-09-30T01:01:31.614 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-09-30T01:01:31.843 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi150.front.sepia.ceph.com 2023-09-30T01:01:31.844 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-09-30T01:01:32.056 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi150.front.sepia.ceph.com 2023-09-30T01:01:32.056 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-09-30T01:01:32.235 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi150.front.sepia.ceph.com 2023-09-30T01:01:32.235 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-09-30T01:01:32.416 INFO:tasks.ceph:Archiving mon data... 2023-09-30T01:01:32.418 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794/data/mon.a.tgz 2023-09-30T01:01:32.418 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2023-09-30T01:01:32.424 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.p3TjJZJU5h 2023-09-30T01:01:32.424 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.p3TjJZJU5h 2023-09-30T01:01:32.501 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.p3TjJZJU5h 2023-09-30T01:01:32.522 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.p3TjJZJU5h is 292KB 2023-09-30T01:01:32.584 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.p3TjJZJU5h 2023-09-30T01:01:32.591 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794/data/mon.c.tgz 2023-09-30T01:01:32.591 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2023-09-30T01:01:32.598 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.3WfGkIRV2g 2023-09-30T01:01:32.598 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.3WfGkIRV2g 2023-09-30T01:01:32.718 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.3WfGkIRV2g 2023-09-30T01:01:32.740 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.3WfGkIRV2g is 292KB 2023-09-30T01:01:32.796 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.3WfGkIRV2g 2023-09-30T01:01:32.802 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794/data/mon.b.tgz 2023-09-30T01:01:32.803 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2023-09-30T01:01:32.809 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.ceURaM0a0l 2023-09-30T01:01:32.809 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ceURaM0a0l 2023-09-30T01:01:32.923 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.ceURaM0a0l 2023-09-30T01:01:32.945 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.ceURaM0a0l is 292KB 2023-09-30T01:01:33.006 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.ceURaM0a0l 2023-09-30T01:01:33.013 INFO:tasks.ceph:Cleaning ceph cluster... 2023-09-30T01:01:33.013 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-30T01:01:33.016 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-30T01:01:33.032 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-09-30T01:01:33.032 DEBUG:teuthology.orchestra.run.smithi069:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-30T01:01:33.071 DEBUG:teuthology.orchestra.run.smithi150:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-30T01:01:33.092 INFO:teuthology.orchestra.run.smithi069.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-30T01:01:33.107 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-30T01:01:33.114 INFO:tasks.ceph:Archiving crash dumps... 2023-09-30T01:01:33.115 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794/remote/smithi069/crash 2023-09-30T01:01:33.116 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-30T01:01:33.134 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794/remote/smithi150/crash 2023-09-30T01:01:33.135 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-30T01:01:33.152 INFO:tasks.ceph:Compressing logs... 2023-09-30T01:01:33.152 DEBUG:teuthology.orchestra.run.smithi069:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T01:01:33.179 DEBUG:teuthology.orchestra.run.smithi150:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T01:01:34.202 INFO:tasks.ceph:Archiving logs... 2023-09-30T01:01:34.203 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794/remote/smithi069/log 2023-09-30T01:01:34.204 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T01:01:34.490 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794/remote/smithi150/log 2023-09-30T01:01:34.491 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T01:01:34.641 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1895, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1472, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph_manager.py", line 3189, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-09-30T01:01:34.711 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=12c407d0829e4ceeb46c516d0966a3ec Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1895, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1472, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph_manager.py", line 3189, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-09-30T01:01:34.714 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-09-30T01:01:34.740 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-30T01:01:34.776 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-30T01:01:34.777 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T01:01:34.789 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T01:01:34.859 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-30T01:01:34.868 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-30T01:01:34.868 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T01:01:34.871 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T01:01:34.890 INFO:teuthology.orchestra.run.smithi069.stdout: remote refid st t when poll reach delay offset jitter 2023-09-30T01:01:34.890 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================== 2023-09-30T01:01:34.891 INFO:teuthology.orchestra.run.smithi069.stdout: hv01.front.sepi 44.4.53.2 2 u 55 64 377 0.058 1.122 0.083 2023-09-30T01:01:34.891 INFO:teuthology.orchestra.run.smithi069.stdout: hv02.front.sepi 63.231.80.2 3 u 53 64 377 0.090 1.193 0.073 2023-09-30T01:01:34.891 INFO:teuthology.orchestra.run.smithi069.stdout:*hv03.front.sepi 74.6.168.72 3 u 56 64 377 0.089 0.146 0.382 2023-09-30T01:01:34.891 INFO:teuthology.orchestra.run.smithi069.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T01:01:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: remote refid st t when poll reach delay offset jitter 2023-09-30T01:01:34.894 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================== 2023-09-30T01:01:34.894 INFO:teuthology.orchestra.run.smithi150.stdout:*hv01.front.sepi 44.4.53.2 2 u 56 64 377 0.088 1.351 0.962 2023-09-30T01:01:34.894 INFO:teuthology.orchestra.run.smithi150.stdout:+hv02.front.sepi 63.231.80.2 3 u 55 64 377 0.062 0.342 0.787 2023-09-30T01:01:34.894 INFO:teuthology.orchestra.run.smithi150.stdout:+hv03.front.sepi 74.6.168.72 3 u 50 64 377 0.079 0.010 0.688 2023-09-30T01:01:34.894 INFO:teuthology.orchestra.run.smithi150.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-30T01:01:34.895 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-30T01:01:34.904 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-30T01:01:34.905 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-30T01:01:34.927 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-30T01:01:34.952 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-30T01:01:34.986 INFO:teuthology.task.internal:Duration was 862.616292 seconds 2023-09-30T01:01:34.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-30T01:01:35.018 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-30T01:01:35.019 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T01:01:35.021 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T01:01:35.073 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-30T01:01:35.074 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2023-09-30T01:01:35.074 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T01:01:35.085 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2023-09-30T01:01:35.086 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T01:01:35.098 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-30T01:01:35.099 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T01:01:35.101 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T01:01:35.133 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-30T01:01:35.133 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T01:01:35.176 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T01:01:35.241 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-30T01:01:35.253 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-30T01:01:35.253 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T01:01:35.270 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T01:01:35.287 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-30T01:01:35.296 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T01:01:35.299 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T01:01:35.311 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2023-09-30T01:01:35.341 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2023-09-30T01:01:35.371 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T01:01:35.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:01:35.384 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T01:01:35.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T01:01:35.390 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-30T01:01:35.404 INFO:teuthology.task.internal:Transferring archived files... 2023-09-30T01:01:35.405 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794/remote/smithi069 2023-09-30T01:01:35.405 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-30T01:01:35.430 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794/remote/smithi150 2023-09-30T01:01:35.430 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-30T01:01:35.453 INFO:teuthology.task.internal:Removing archive directory... 2023-09-30T01:01:35.453 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-30T01:01:35.455 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-30T01:01:35.502 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-30T01:01:35.513 INFO:teuthology.task.internal:Not uploading archives. 2023-09-30T01:01:35.513 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-30T01:01:35.536 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-30T01:01:35.536 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T01:01:35.539 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T01:01:35.544 INFO:teuthology.orchestra.run.smithi069.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 01:01 /home/ubuntu/cephtest 2023-09-30T01:01:35.547 INFO:teuthology.orchestra.run.smithi150.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 01:01 /home/ubuntu/cephtest 2023-09-30T01:01:35.549 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-30T01:01:35.570 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-30T01:01:35.637 INFO:teuthology.nuke:Checking targets against current locks 2023-09-30T01:01:35.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-09-30T01:01:35.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-09-30T01:01:35.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-09-30T01:01:35.657 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-09-30T01:01:35.657 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-09-30T01:01:35.657 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-09-30T01:01:35.665 DEBUG:teuthology.nuke:shortname: smithi069 2023-09-30T01:01:35.665 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T01:01:35.689 DEBUG:teuthology.nuke:shortname: smithi150 2023-09-30T01:01:35.690 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T01:01:35.695 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 00:38:16.726695', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-09-30T01:01:35.717 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406794', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 00:38:16.728600', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-09-30T01:01:35.738 INFO:teuthology.orchestra.console.smithi069:Power off 2023-09-30T01:01:35.739 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T01:01:35.765 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2023-09-30T01:01:35.766 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:01:35.790 INFO:teuthology.orchestra.console.smithi150:Power off 2023-09-30T01:01:35.790 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T01:01:35.802 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-09-30T01:01:35.817 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2023-09-30T01:01:35.818 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:01:35.841 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-09-30T01:01:39.803 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:01:39.842 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:01:39.912 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-09-30T01:01:39.952 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-09-30T01:01:43.914 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:01:43.953 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:01:44.027 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-09-30T01:01:44.065 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-09-30T01:01:48.030 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:01:48.066 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T01:01:48.140 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2023-09-30T01:01:48.176 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2023-09-30T01:01:48.241 INFO:teuthology.orchestra.console.smithi069:Power off completed 2023-09-30T01:01:48.277 INFO:teuthology.orchestra.console.smithi150:Power off completed 2023-09-30T01:01:48.383 INFO:teuthology.run:Summary data: description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-snaps} duration: 862.616292476654 failure_reason: '"1696035346.422124 mgr.y (mgr.4101) 1 : cluster [ERR] Unhandled exception from module ''pg_autoscaler'' while running on mgr.y: ''PgAutoscaler'' object has no attribute ''noautoscale''" in cluster log' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=12c407d0829e4ceeb46c516d0966a3ec status: fail success: false 2023-09-30T01:01:48.383 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T01:01:48.470 INFO:teuthology.run:FAIL