2023-01-25T05:25:43.711 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-25T05:25:43.712 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-snaps} --name yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi --owner scheduled_yuriw@teuthology --verbose --archive /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136832 -- /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136832/orig.config.yaml 2023-01-25T05:25:43.756 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-25T05:25:43.831 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136832 branch: wip-yuri-testing-2023-01-23-0926 description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-snaps} email: yweinstein@radhat.com first_in_suite: false job_id: '7136832' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri-testing-2023-01-23-0926 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 ms type: async osd_max_pg_log_entries: 2 osd_min_pg_log_entries: 1 osd_pg_log_trim_min: 0 osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore odsync write: true mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd backoff on degraded: true osd backoff on peering: true osd blocked scrub grace period: 3600 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 2 osd max markdown count: 1000 osd max scrubs: 3 osd mclock override recovery settings: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 0.5 osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - must scrub before tier agent can activate sha1: b0d2592529629ab4f86fed5991e556a87dcda563 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: b0d2592529629ab4f86fed5991e556a87dcda563 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 min_in: 4 workunit: branch: wip-yuri-testing-2023-01-23-0926 sha1: b0d2592529629ab4f86fed5991e556a87dcda563 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: b0d2592529629ab4f86fed5991e556a87dcda563 sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri-testing-2023-01-23-0926 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b0d2592529629ab4f86fed5991e556a87dcda563 targets: smithi077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 2 chance_pgpnum_fix: 1 timeout: 1200 - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache target_max_objects 250 - sudo ceph osd pool set cache min_read_recency_for_promote 2 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_22:20:59 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-25T05:25:43.832 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa; will attempt to use it 2023-01-25T05:25:43.833 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa/tasks 2023-01-25T05:25:43.833 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-25T05:25:43.835 INFO:teuthology.task.internal:Checking packages... 2023-01-25T05:25:43.864 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'b0d2592529629ab4f86fed5991e556a87dcda563' 2023-01-25T05:25:43.864 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-25T05:25:43.865 INFO:teuthology.packaging:ref: None 2023-01-25T05:25:43.865 INFO:teuthology.packaging:tag: None 2023-01-25T05:25:43.865 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-01-23-0926 2023-01-25T05:25:43.866 INFO:teuthology.packaging:sha1: b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:25:43.866 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-01-23-0926 2023-01-25T05:25:44.019 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1959.gb0d25925 2023-01-25T05:25:44.019 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-25T05:25:44.028 INFO:teuthology.task.internal:no buildpackages task found 2023-01-25T05:25:44.029 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-25T05:25:44.036 INFO:teuthology.task.internal:Saving configuration 2023-01-25T05:25:44.054 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-25T05:25:44.062 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-25T05:25:44.096 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136832', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 05:20:44.357564', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-01-25T05:25:44.125 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136832', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 05:20:44.359172', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-01-25T05:25:44.126 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-25T05:25:44.134 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-25T05:25:44.134 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-25T05:25:44.135 INFO:teuthology.run_tasks:Running task console_log... 2023-01-25T05:25:44.212 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f77f4bc5670>, signals=[15]) 2023-01-25T05:25:44.214 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-25T05:25:44.221 INFO:teuthology.task.internal:Opening connections... 2023-01-25T05:25:44.222 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2023-01-25T05:25:44.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T05:25:44.293 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2023-01-25T05:25:44.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T05:25:44.360 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-25T05:25:44.368 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2023-01-25T05:25:44.389 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2023-01-25T05:25:44.390 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2023-01-25T05:25:44.449 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="Red Hat Enterprise Linux" 2023-01-25T05:25:44.449 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="8.6 (Ootpa)" 2023-01-25T05:25:44.449 INFO:teuthology.orchestra.run.smithi077.stdout:ID="rhel" 2023-01-25T05:25:44.450 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="fedora" 2023-01-25T05:25:44.450 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="8.6" 2023-01-25T05:25:44.450 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el8" 2023-01-25T05:25:44.450 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-25T05:25:44.450 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2023-01-25T05:25:44.451 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-25T05:25:44.451 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://www.redhat.com/" 2023-01-25T05:25:44.451 INFO:teuthology.orchestra.run.smithi077.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-25T05:25:44.451 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T05:25:44.451 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:25:44.452 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T05:25:44.452 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-25T05:25:44.452 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-25T05:25:44.452 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-25T05:25:44.453 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2023-01-25T05:25:44.478 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2023-01-25T05:25:44.498 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2023-01-25T05:25:44.499 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2023-01-25T05:25:44.559 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Red Hat Enterprise Linux" 2023-01-25T05:25:44.559 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8.6 (Ootpa)" 2023-01-25T05:25:44.560 INFO:teuthology.orchestra.run.smithi121.stdout:ID="rhel" 2023-01-25T05:25:44.560 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="fedora" 2023-01-25T05:25:44.560 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8.6" 2023-01-25T05:25:44.560 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2023-01-25T05:25:44.561 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-25T05:25:44.561 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2023-01-25T05:25:44.561 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-25T05:25:44.561 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.redhat.com/" 2023-01-25T05:25:44.561 INFO:teuthology.orchestra.run.smithi121.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-25T05:25:44.562 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T05:25:44.562 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:25:44.562 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T05:25:44.562 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-25T05:25:44.562 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-25T05:25:44.562 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-25T05:25:44.563 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2023-01-25T05:25:44.581 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-25T05:25:44.592 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-25T05:25:44.600 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-25T05:25:44.601 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2023-01-25T05:25:44.604 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2023-01-25T05:25:44.622 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-25T05:25:44.631 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-25T05:25:44.631 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2023-01-25T05:25:44.663 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2023-01-25T05:25:44.697 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-25T05:25:44.760 INFO:teuthology.run_tasks:Running task kernel... 2023-01-25T05:25:44.779 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-25T05:25:44.779 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-25T05:25:44.780 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T05:25:44.781 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T05:25:44.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T05:25:44.800 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2023-01-25T05:25:44.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T05:25:44.801 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2023-01-25T05:25:44.859 INFO:teuthology.orchestra.run.smithi121.stdout:4.18.0-372.9.1.el8.x86_64 2023-01-25T05:25:44.860 INFO:teuthology.task.kernel:Running kernel on smithi121: 4.18.0-372.9.1.el8.x86_64 2023-01-25T05:25:44.860 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum install -y kernel 2023-01-25T05:25:44.861 INFO:teuthology.orchestra.run.smithi077.stdout:4.18.0-372.9.1.el8.x86_64 2023-01-25T05:25:44.862 INFO:teuthology.task.kernel:Running kernel on smithi077: 4.18.0-372.9.1.el8.x86_64 2023-01-25T05:25:44.862 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum install -y kernel 2023-01-25T05:25:45.966 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:25:45.966 INFO:teuthology.orchestra.run.smithi121.stdout:Unable to read consumer identity 2023-01-25T05:25:46.074 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:25:46.074 INFO:teuthology.orchestra.run.smithi077.stdout:Unable to read consumer identity 2023-01-25T05:25:47.456 INFO:teuthology.orchestra.run.smithi077.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-01-25T05:25:47.459 INFO:teuthology.orchestra.run.smithi121.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-01-25T05:25:48.440 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2023-01-25T05:25:48.467 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:00 2023-01-25T05:25:51.432 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 273 kB/s | 24 kB 00:00 2023-01-25T05:25:51.509 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 391 kB/s | 24 kB 00:00 2023-01-25T05:25:52.084 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-01-25T05:25:52.135 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:25:52.136 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2023-01-25T05:25:52.137 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:25:52.153 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-01-25T05:25:52.206 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:25:52.206 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-25T05:25:52.207 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:25:52.274 DEBUG:teuthology.orchestra.run.smithi077:> echo no | sudo yum reinstall kernel || true 2023-01-25T05:25:52.339 DEBUG:teuthology.orchestra.run.smithi121:> echo no | sudo yum reinstall kernel || true 2023-01-25T05:25:52.643 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:25:52.643 INFO:teuthology.orchestra.run.smithi077.stdout:Unable to read consumer identity 2023-01-25T05:25:52.711 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:25:52.711 INFO:teuthology.orchestra.run.smithi121.stdout:Unable to read consumer identity 2023-01-25T05:25:52.901 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:01 ago on Wed 25 Jan 2023 05:25:51 AM UTC. 2023-01-25T05:25:52.947 INFO:teuthology.orchestra.run.smithi077.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T05:25:52.964 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:01 ago on Wed 25 Jan 2023 05:25:51 AM UTC. 2023-01-25T05:25:53.000 INFO:teuthology.orchestra.run.smithi077.stderr:Error: No packages marked for reinstall. 2023-01-25T05:25:53.011 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T05:25:53.040 INFO:teuthology.orchestra.run.smithi121.stderr:Error: No packages marked for reinstall. 2023-01-25T05:25:53.045 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum reinstall -y kernel || true 2023-01-25T05:25:53.086 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum reinstall -y kernel || true 2023-01-25T05:25:53.373 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:25:53.373 INFO:teuthology.orchestra.run.smithi077.stdout:Unable to read consumer identity 2023-01-25T05:25:53.414 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:25:53.414 INFO:teuthology.orchestra.run.smithi121.stdout:Unable to read consumer identity 2023-01-25T05:25:53.633 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Jan 2023 05:25:51 AM UTC. 2023-01-25T05:25:53.662 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Jan 2023 05:25:51 AM UTC. 2023-01-25T05:25:53.674 INFO:teuthology.orchestra.run.smithi077.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T05:25:53.697 INFO:teuthology.orchestra.run.smithi077.stderr:Error: No packages marked for reinstall. 2023-01-25T05:25:53.702 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T05:25:53.721 INFO:teuthology.orchestra.run.smithi121.stderr:Error: No packages marked for reinstall. 2023-01-25T05:25:53.743 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q kernel --last | head -n 1 2023-01-25T05:25:53.766 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q kernel --last | head -n 1 2023-01-25T05:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-01-25T05:25:53.824 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-01-25T05:25:53.824 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-25T05:25:53.825 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-25T05:25:53.825 DEBUG:teuthology.orchestra.run.smithi077:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T05:25:53.844 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-01-25T05:25:53.846 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-01-25T05:25:53.846 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-25T05:25:53.847 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-25T05:25:53.847 DEBUG:teuthology.orchestra.run.smithi121:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T05:25:53.851 INFO:teuthology.orchestra.run.smithi077.stdout:ttyS1 2023-01-25T05:25:53.873 DEBUG:teuthology.parallel:result is None 2023-01-25T05:25:53.887 INFO:teuthology.orchestra.run.smithi121.stdout:ttyS1 2023-01-25T05:25:53.904 DEBUG:teuthology.parallel:result is None 2023-01-25T05:25:53.904 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-25T05:25:53.915 INFO:teuthology.task.internal:Creating test directory... 2023-01-25T05:25:53.915 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T05:25:53.919 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T05:25:53.960 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-25T05:25:53.971 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-25T05:25:53.986 INFO:teuthology.task.internal:Creating archive directory... 2023-01-25T05:25:53.986 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T05:25:53.989 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T05:25:54.022 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-25T05:25:54.034 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-25T05:25:54.034 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T05:25:54.046 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T05:25:54.095 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T05:25:54.096 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T05:25:54.106 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T05:25:54.106 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T05:25:54.109 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-25T05:25:54.120 INFO:teuthology.task.internal:Configuring sudo... 2023-01-25T05:25:54.121 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T05:25:54.123 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T05:25:54.176 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-25T05:25:54.186 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-25T05:25:54.187 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T05:25:54.190 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T05:25:54.232 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T05:25:54.294 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T05:25:54.338 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T05:25:54.420 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T05:25:54.462 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:25:54.462 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T05:25:54.529 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T05:25:54.571 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T05:25:54.623 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T05:25:54.707 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T05:25:54.748 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:25:54.749 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T05:25:54.816 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2023-01-25T05:25:54.819 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2023-01-25T05:25:54.905 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T05:25:54.910 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T05:25:55.383 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-25T05:25:55.396 INFO:teuthology.task.internal:Starting timer... 2023-01-25T05:25:55.397 INFO:teuthology.run_tasks:Running task pcp... 2023-01-25T05:25:55.419 INFO:teuthology.run_tasks:Running task selinux... 2023-01-25T05:25:55.433 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-25T05:25:55.433 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2023-01-25T05:25:55.493 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2023-01-25T05:25:55.494 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2023-01-25T05:25:55.559 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2023-01-25T05:25:55.562 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-25T05:25:55.562 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2023-01-25T05:25:55.587 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2023-01-25T05:25:55.587 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2023-01-25T05:25:55.605 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2023-01-25T05:25:55.605 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi077.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi121.front.sepia.ceph.com': 'permissive'} 2023-01-25T05:25:55.606 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T05:25:55.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T05:25:55.656 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T05:25:55.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T05:25:55.683 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-25T05:25:55.683 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2023-01-25T05:25:55.738 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2023-01-25T05:25:55.764 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-25T05:25:55.777 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-01-25T05:25:55.782 INFO:teuthology.repo_utils:Fetching main from origin 2023-01-25T05:25:55.891 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-25T05:25:55.907 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-25T05:25:55.908 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi077.front.sepia.ceph.com,smithi121.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-25T05:32:39.716 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi077.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com')] 2023-01-25T05:32:39.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-01-25T05:32:39.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T05:32:39.787 DEBUG:teuthology.orchestra.run.smithi077:> true 2023-01-25T05:32:39.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-01-25T05:32:39.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T05:32:39.922 DEBUG:teuthology.orchestra.run.smithi121:> true 2023-01-25T05:32:39.991 INFO:teuthology.run_tasks:Running task clock... 2023-01-25T05:32:40.002 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-25T05:32:40.002 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T05:32:40.003 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T05:32:40.005 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T05:32:40.006 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T05:32:40.035 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T05:32:40.049 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T05:32:40.065 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T05:32:40.079 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T05:32:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2023-01-25T05:32:40.136 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2023-01-25T05:32:40.150 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T05:32:40.158 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2023-01-25T05:32:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T05:32:40.170 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2023-01-25T05:32:40.184 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T05:32:40.198 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T05:32:40.237 INFO:teuthology.orchestra.run.smithi077.stderr:bash: ntpq: command not found 2023-01-25T05:32:40.240 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T05:32:40.241 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2023-01-25T05:32:40.241 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:32:40.241 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:32:40.241 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:32:40.241 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:32:40.264 INFO:teuthology.orchestra.run.smithi121.stderr:bash: ntpq: command not found 2023-01-25T05:32:40.268 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T05:32:40.268 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2023-01-25T05:32:40.268 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:32:40.269 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:32:40.269 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:32:40.269 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T05:32:40.270 INFO:teuthology.run_tasks:Running task install... 2023-01-25T05:32:40.283 DEBUG:teuthology.task.install:project ceph 2023-01-25T05:32:40.284 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}} 2023-01-25T05:32:40.285 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'} 2023-01-25T05:32:40.285 INFO:teuthology.task.install:Using flavor: default 2023-01-25T05:32:40.297 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-25T05:32:40.297 INFO:teuthology.task.install:extra packages: [] 2023-01-25T05:32:40.298 DEBUG:teuthology.orchestra.run.smithi077:> sudo lsb_release -is 2023-01-25T05:32:40.339 INFO:teuthology.orchestra.run.smithi077.stdout:RedHatEnterprise 2023-01-25T05:32:40.341 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:32:40.341 DEBUG:teuthology.orchestra.run.smithi121:> sudo lsb_release -is 2023-01-25T05:32:40.342 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'tag': None, 'wait_for_package': False} 2023-01-25T05:32:40.342 DEBUG:teuthology.orchestra.run.smithi077:> sudo lsb_release -is 2023-01-25T05:32:40.384 INFO:teuthology.orchestra.run.smithi121.stdout:RedHatEnterprise 2023-01-25T05:32:40.385 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:32:40.385 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'tag': None, 'wait_for_package': False} 2023-01-25T05:32:40.386 DEBUG:teuthology.orchestra.run.smithi121:> sudo lsb_release -is 2023-01-25T05:32:40.421 INFO:teuthology.orchestra.run.smithi077.stdout:RedHatEnterprise 2023-01-25T05:32:40.422 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:32:40.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:32:40.465 INFO:teuthology.orchestra.run.smithi121.stdout:RedHatEnterprise 2023-01-25T05:32:40.466 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:32:40.467 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:32:40.579 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/ 2023-01-25T05:32:40.579 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1959.gb0d25925 2023-01-25T05:32:40.618 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/ 2023-01-25T05:32:40.618 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1959.gb0d25925 2023-01-25T05:32:40.702 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T05:32:40.702 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:32:40.703 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T05:32:40.729 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T05:32:40.730 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0d2592529629ab4f86fed5991e556a87dcda563/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T05:32:40.732 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T05:32:40.732 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:32:40.733 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T05:32:40.759 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T05:32:40.759 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0d2592529629ab4f86fed5991e556a87dcda563/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T05:32:40.797 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T05:32:40.825 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T05:32:40.874 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T05:32:40.902 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T05:32:40.940 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2023-01-25T05:32:40.942 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2023-01-25T05:32:40.968 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2023-01-25T05:32:40.969 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2023-01-25T05:32:41.311 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:32:41.339 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:32:41.554 INFO:teuthology.orchestra.run.smithi077.stdout:44 files removed 2023-01-25T05:32:41.571 INFO:teuthology.orchestra.run.smithi121.stdout:44 files removed 2023-01-25T05:32:41.600 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw 2023-01-25T05:32:41.617 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw 2023-01-25T05:32:41.928 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:32:41.947 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:32:42.562 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 300 kB/s | 80 kB 00:00 2023-01-25T05:32:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 273 kB/s | 80 kB 00:00 2023-01-25T05:32:42.837 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 38 kB/s | 9.3 kB 00:00 2023-01-25T05:32:42.884 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 37 kB/s | 9.3 kB 00:00 2023-01-25T05:32:43.096 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 6.3 kB/s | 1.5 kB 00:00 2023-01-25T05:32:43.133 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2023-01-25T05:32:43.248 INFO:teuthology.orchestra.run.smithi077.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-01-25T05:32:43.276 INFO:teuthology.orchestra.run.smithi121.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-01-25T05:32:44.239 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2023-01-25T05:32:44.318 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:01 2023-01-25T05:32:47.310 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 415 kB/s | 24 kB 00:00 2023-01-25T05:32:47.314 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 203 kB/s | 24 kB 00:00 2023-01-25T05:32:48.211 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-01-25T05:32:48.216 INFO:teuthology.orchestra.run.smithi077.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-01-25T05:32:56.203 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 63 MB/s | 47 MB 00:00 2023-01-25T05:32:56.450 INFO:teuthology.orchestra.run.smithi077.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 47 MB 00:01 2023-01-25T05:33:07.082 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:33:07.085 INFO:teuthology.orchestra.run.smithi121.stdout:==================================================================================================== 2023-01-25T05:33:07.085 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-25T05:33:07.086 INFO:teuthology.orchestra.run.smithi121.stdout:==================================================================================================== 2023-01-25T05:33:07.086 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:33:07.086 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 13 M 2023-01-25T05:33:07.086 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2023-01-25T05:33:07.086 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T05:33:07.087 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T05:33:07.087 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-25T05:33:07.087 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 5.2 M 2023-01-25T05:33:07.087 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 M 2023-01-25T05:33:07.087 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 k 2023-01-25T05:33:07.088 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T05:33:07.088 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 768 k 2023-01-25T05:33:07.088 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T05:33:07.088 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-25T05:33:07.088 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 474 k 2023-01-25T05:33:07.089 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-01-25T05:33:07.089 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.5 M 2023-01-25T05:33:07.089 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T05:33:07.089 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-25T05:33:07.089 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 45 k 2023-01-25T05:33:07.090 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 115 k 2023-01-25T05:33:07.090 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 190 k 2023-01-25T05:33:07.090 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-25T05:33:07.090 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 372 k 2023-01-25T05:33:07.090 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 364 k 2023-01-25T05:33:07.091 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 107 k 2023-01-25T05:33:07.091 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T05:33:07.091 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:33:07.091 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:33:07.091 INFO:teuthology.orchestra.run.smithi121.stdout:==================================================================================================== 2023-01-25T05:33:07.092 INFO:teuthology.orchestra.run.smithi121.stdout:Install 21 Packages 2023-01-25T05:33:07.092 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 2 Packages 2023-01-25T05:33:07.092 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:33:07.093 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 62 M 2023-01-25T05:33:07.093 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:33:07.302 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:33:07.304 INFO:teuthology.orchestra.run.smithi077.stdout:==================================================================================================== 2023-01-25T05:33:07.304 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-01-25T05:33:07.305 INFO:teuthology.orchestra.run.smithi077.stdout:==================================================================================================== 2023-01-25T05:33:07.305 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:33:07.305 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 13 M 2023-01-25T05:33:07.305 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2023-01-25T05:33:07.305 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T05:33:07.306 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T05:33:07.306 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2023-01-25T05:33:07.306 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 5.2 M 2023-01-25T05:33:07.306 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 M 2023-01-25T05:33:07.306 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 k 2023-01-25T05:33:07.307 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T05:33:07.307 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 768 k 2023-01-25T05:33:07.307 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T05:33:07.307 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-25T05:33:07.307 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 474 k 2023-01-25T05:33:07.308 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-01-25T05:33:07.308 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.5 M 2023-01-25T05:33:07.308 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T05:33:07.308 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-25T05:33:07.308 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 45 k 2023-01-25T05:33:07.309 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 115 k 2023-01-25T05:33:07.309 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 190 k 2023-01-25T05:33:07.309 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-25T05:33:07.309 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 372 k 2023-01-25T05:33:07.310 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 364 k 2023-01-25T05:33:07.310 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 107 k 2023-01-25T05:33:07.310 INFO:teuthology.orchestra.run.smithi077.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T05:33:07.310 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:33:07.310 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:33:07.311 INFO:teuthology.orchestra.run.smithi077.stdout:==================================================================================================== 2023-01-25T05:33:07.311 INFO:teuthology.orchestra.run.smithi077.stdout:Install 21 Packages 2023-01-25T05:33:07.311 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 2 Packages 2023-01-25T05:33:07.311 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:33:07.312 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 62 M 2023-01-25T05:33:07.312 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:33:07.894 INFO:teuthology.orchestra.run.smithi121.stdout:(1/23): ceph-base-18.0.0-1959.gb0d25925.el8.x86 6.5 MB/s | 5.2 MB 00:00 2023-01-25T05:33:08.328 INFO:teuthology.orchestra.run.smithi121.stdout:(2/23): ceph-selinux-18.0.0-1959.gb0d25925.el8. 56 kB/s | 24 kB 00:00 2023-01-25T05:33:08.967 INFO:teuthology.orchestra.run.smithi077.stdout:(1/23): ceph-base-18.0.0-1959.gb0d25925.el8.x86 3.1 MB/s | 5.2 MB 00:01 2023-01-25T05:33:09.011 INFO:teuthology.orchestra.run.smithi121.stdout:(3/23): ceph-radosgw-18.0.0-1959.gb0d25925.el8. 6.8 MB/s | 13 MB 00:01 2023-01-25T05:33:09.026 INFO:teuthology.orchestra.run.smithi077.stdout:(2/23): ceph-selinux-18.0.0-1959.gb0d25925.el8. 418 kB/s | 24 kB 00:00 2023-01-25T05:33:09.095 INFO:teuthology.orchestra.run.smithi121.stdout:(4/23): libradosstriper1-18.0.0-1959.gb0d25925. 5.6 MB/s | 474 kB 00:00 2023-01-25T05:33:09.235 INFO:teuthology.orchestra.run.smithi077.stdout:(3/23): libcephfs2-18.0.0-1959.gb0d25925.el8.x8 3.6 MB/s | 768 kB 00:00 2023-01-25T05:33:09.410 INFO:teuthology.orchestra.run.smithi077.stdout:(4/23): libradosstriper1-18.0.0-1959.gb0d25925. 2.6 MB/s | 474 kB 00:00 2023-01-25T05:33:09.420 INFO:teuthology.orchestra.run.smithi121.stdout:(5/23): libcephfs2-18.0.0-1959.gb0d25925.el8.x8 703 kB/s | 768 kB 00:01 2023-01-25T05:33:09.520 INFO:teuthology.orchestra.run.smithi121.stdout:(6/23): python3-ceph-argparse-18.0.0-1959.gb0d2 446 kB/s | 45 kB 00:00 2023-01-25T05:33:09.594 INFO:teuthology.orchestra.run.smithi077.stdout:(5/23): ceph-radosgw-18.0.0-1959.gb0d25925.el8. 5.7 MB/s | 13 MB 00:02 2023-01-25T05:33:09.669 INFO:teuthology.orchestra.run.smithi077.stdout:(6/23): python3-ceph-argparse-18.0.0-1959.gb0d2 596 kB/s | 45 kB 00:00 2023-01-25T05:33:09.720 INFO:teuthology.orchestra.run.smithi121.stdout:(7/23): python3-ceph-common-18.0.0-1959.gb0d259 574 kB/s | 115 kB 00:00 2023-01-25T05:33:09.752 INFO:teuthology.orchestra.run.smithi077.stdout:(7/23): python3-ceph-common-18.0.0-1959.gb0d259 1.3 MB/s | 115 kB 00:00 2023-01-25T05:33:09.811 INFO:teuthology.orchestra.run.smithi077.stdout:(8/23): python3-cephfs-18.0.0-1959.gb0d25925.el 3.2 MB/s | 190 kB 00:00 2023-01-25T05:33:09.812 INFO:teuthology.orchestra.run.smithi121.stdout:(8/23): librgw2-18.0.0-1959.gb0d25925.el8.x86_6 9.0 MB/s | 6.5 MB 00:00 2023-01-25T05:33:09.878 INFO:teuthology.orchestra.run.smithi077.stdout:(9/23): python3-rados-18.0.0-1959.gb0d25925.el8 5.4 MB/s | 372 kB 00:00 2023-01-25T05:33:09.904 INFO:teuthology.orchestra.run.smithi121.stdout:(9/23): python3-rados-18.0.0-1959.gb0d25925.el8 4.0 MB/s | 372 kB 00:00 2023-01-25T05:33:09.936 INFO:teuthology.orchestra.run.smithi077.stdout:(10/23): python3-rbd-18.0.0-1959.gb0d25925.el8. 6.1 MB/s | 364 kB 00:00 2023-01-25T05:33:09.987 INFO:teuthology.orchestra.run.smithi077.stdout:(11/23): python3-rgw-18.0.0-1959.gb0d25925.el8. 2.1 MB/s | 107 kB 00:00 2023-01-25T05:33:10.171 INFO:teuthology.orchestra.run.smithi121.stdout:(10/23): ceph-common-18.0.0-1959.gb0d25925.el8. 7.9 MB/s | 24 MB 00:03 2023-01-25T05:33:10.197 INFO:teuthology.orchestra.run.smithi121.stdout:(11/23): python3-cephfs-18.0.0-1959.gb0d25925.e 400 kB/s | 190 kB 00:00 2023-01-25T05:33:10.212 INFO:teuthology.orchestra.run.smithi077.stdout:(12/23): librgw2-18.0.0-1959.gb0d25925.el8.x86_ 8.1 MB/s | 6.5 MB 00:00 2023-01-25T05:33:10.221 INFO:teuthology.orchestra.run.smithi121.stdout:(12/23): python3-rgw-18.0.0-1959.gb0d25925.el8. 2.1 MB/s | 107 kB 00:00 2023-01-25T05:33:10.237 INFO:teuthology.orchestra.run.smithi077.stdout:(13/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-01-25T05:33:10.247 INFO:teuthology.orchestra.run.smithi121.stdout:(13/23): python3-rbd-18.0.0-1959.gb0d25925.el8. 1.0 MB/s | 364 kB 00:00 2023-01-25T05:33:10.279 INFO:teuthology.orchestra.run.smithi077.stdout:(14/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2023-01-25T05:33:10.364 INFO:teuthology.orchestra.run.smithi121.stdout:(14/23): liboath-2.6.2-3.el8.x86_64.rpm 414 kB/s | 59 kB 00:00 2023-01-25T05:33:10.380 INFO:teuthology.orchestra.run.smithi077.stdout:(15/23): liboath-2.6.2-3.el8.x86_64.rpm 352 kB/s | 59 kB 00:00 2023-01-25T05:33:10.389 INFO:teuthology.orchestra.run.smithi121.stdout:(15/23): libunwind-1.3.1-3.el8.x86_64.rpm 529 kB/s | 75 kB 00:00 2023-01-25T05:33:10.423 INFO:teuthology.orchestra.run.smithi121.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-01-25T05:33:10.463 INFO:teuthology.orchestra.run.smithi077.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 563 kB/s | 47 kB 00:00 2023-01-25T05:33:10.497 INFO:teuthology.orchestra.run.smithi077.stdout:(17/23): thrift-0.13.0-2.el8.x86_64.rpm 8.0 MB/s | 1.7 MB 00:00 2023-01-25T05:33:10.522 INFO:teuthology.orchestra.run.smithi077.stdout:(18/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-01-25T05:33:10.581 INFO:teuthology.orchestra.run.smithi077.stdout:(19/23): python3-prettytable-0.7.2-14.el8.noarc 527 kB/s | 44 kB 00:00 2023-01-25T05:33:10.606 INFO:teuthology.orchestra.run.smithi077.stdout:(20/23): librdkafka-0.11.4-3.el8.x86_64.rpm 4.4 MB/s | 354 kB 00:00 2023-01-25T05:33:10.627 INFO:teuthology.orchestra.run.smithi121.stdout:(17/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 197 kB/s | 47 kB 00:00 2023-01-25T05:33:10.657 INFO:teuthology.orchestra.run.smithi121.stdout:(18/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.1 MB/s | 259 kB 00:00 2023-01-25T05:33:10.684 INFO:teuthology.orchestra.run.smithi121.stdout:(19/23): python3-prettytable-0.7.2-14.el8.noarc 796 kB/s | 44 kB 00:00 2023-01-25T05:33:10.708 INFO:teuthology.orchestra.run.smithi121.stdout:(20/23): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-01-25T05:33:10.825 INFO:teuthology.orchestra.run.smithi121.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 3.8 MB/s | 1.7 MB 00:00 2023-01-25T05:33:10.957 INFO:teuthology.orchestra.run.smithi077.stdout:(21/23): librbd1-18.0.0-1959.gb0d25925.el8.x86_ 10 MB/s | 3.6 MB 00:00 2023-01-25T05:33:11.048 INFO:teuthology.orchestra.run.smithi077.stdout:(22/23): librados2-18.0.0-1959.gb0d25925.el8.x8 7.8 MB/s | 3.6 MB 00:00 2023-01-25T05:33:11.092 INFO:teuthology.orchestra.run.smithi121.stdout:(22/23): librbd1-18.0.0-1959.gb0d25925.el8.x86_ 9.3 MB/s | 3.6 MB 00:00 2023-01-25T05:33:12.041 INFO:teuthology.orchestra.run.smithi121.stdout:(23/23): librados2-18.0.0-1959.gb0d25925.el8.x8 2.7 MB/s | 3.6 MB 00:01 2023-01-25T05:33:12.043 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:33:12.044 INFO:teuthology.orchestra.run.smithi121.stdout:Total 12 MB/s | 62 MB 00:04 2023-01-25T05:33:12.091 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:33:12.136 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:33:12.136 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:33:12.472 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:33:12.476 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:33:13.675 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:33:13.913 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T05:33:14.107 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T05:33:14.496 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T05:33:14.723 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:33:14.816 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:33:14.906 INFO:teuthology.orchestra.run.smithi077.stdout:(23/23): ceph-common-18.0.0-1959.gb0d25925.el8. 3.2 MB/s | 24 MB 00:07 2023-01-25T05:33:14.909 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:33:14.909 INFO:teuthology.orchestra.run.smithi077.stdout:Total 8.1 MB/s | 62 MB 00:07 2023-01-25T05:33:14.956 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:33:15.000 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:33:15.001 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:33:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T05:33:15.279 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2023-01-25T05:33:15.346 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:33:15.350 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:33:15.482 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T05:33:15.875 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T05:33:16.048 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:33:16.133 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:33:16.281 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T05:33:16.332 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T05:33:16.521 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/25 2023-01-25T05:33:16.616 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:33:16.667 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 10/25 2023-01-25T05:33:16.826 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T05:33:16.875 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T05:33:17.038 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T05:33:17.107 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T05:33:17.256 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T05:33:17.321 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T05:33:17.435 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T05:33:17.457 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2023-01-25T05:33:17.605 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:33:17.659 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2023-01-25T05:33:17.740 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:33:17.928 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T05:33:18.152 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2023-01-25T05:33:18.329 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T05:33:18.463 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2023-01-25T05:33:18.672 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T05:33:18.730 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T05:33:18.745 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T05:33:18.922 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:33:18.972 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 18/25 2023-01-25T05:33:19.015 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:33:19.155 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T05:33:19.173 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 19/25 2023-01-25T05:33:19.222 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T05:33:19.386 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/25 2023-01-25T05:33:19.532 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 10/25 2023-01-25T05:33:19.732 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T05:33:19.939 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T05:33:20.113 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T05:33:20.172 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T05:33:20.314 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2023-01-25T05:33:20.520 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2023-01-25T05:33:21.439 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2023-01-25T05:33:21.621 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T05:33:21.694 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T05:33:21.912 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 18/25 2023-01-25T05:33:22.105 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 19/25 2023-01-25T05:33:23.000 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:33:23.214 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:33:23.777 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:33:24.061 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T05:33:24.164 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T05:33:24.313 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:33:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:33:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:33:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T05:33:26.893 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T05:33:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T05:33:27.120 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:33:43.093 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:33:43.093 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2023-01-25T05:33:43.094 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2023-01-25T05:33:43.094 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2023-01-25T05:33:43.094 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /run 2023-01-25T05:33:43.094 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2023-01-25T05:33:43.094 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2023-01-25T05:33:43.095 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2023-01-25T05:33:43.095 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2023-01-25T05:33:43.095 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2023-01-25T05:33:43.095 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:33:44.473 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T05:33:44.494 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T05:33:44.494 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T05:33:44.694 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T05:33:44.694 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:33:45.159 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:33:45.159 INFO:teuthology.orchestra.run.smithi121.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-25T05:33:45.159 INFO:teuthology.orchestra.run.smithi121.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-25T05:33:45.159 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:33:45.309 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 1/25 2023-01-25T05:33:45.309 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2/25 2023-01-25T05:33:45.310 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:33:45.310 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T05:33:45.310 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 5/25 2023-01-25T05:33:45.310 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 6/25 2023-01-25T05:33:45.311 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:33:45.311 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 8/25 2023-01-25T05:33:45.311 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 9/25 2023-01-25T05:33:45.311 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 10/25 2023-01-25T05:33:45.311 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T05:33:45.311 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T05:33:45.312 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 13/25 2023-01-25T05:33:45.312 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-25T05:33:45.312 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-25T05:33:45.312 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-25T05:33:45.312 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-25T05:33:45.313 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-25T05:33:45.313 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-01-25T05:33:45.313 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 20/25 2023-01-25T05:33:45.313 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 21/25 2023-01-25T05:33:45.313 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:33:45.314 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T05:33:45.314 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 24/25 2023-01-25T05:33:45.971 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:33:45.972 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2023-01-25T05:33:45.972 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2023-01-25T05:33:45.972 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /dev 2023-01-25T05:33:45.972 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /run 2023-01-25T05:33:45.972 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2023-01-25T05:33:45.973 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2023-01-25T05:33:45.973 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2023-01-25T05:33:45.973 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2023-01-25T05:33:45.973 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /dev 2023-01-25T05:33:45.973 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:33:46.743 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:33:46.744 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:51 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:33:47.349 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T05:33:47.370 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T05:33:47.370 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T05:33:47.570 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T05:33:47.570 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:33:48.012 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:33:48.012 INFO:teuthology.orchestra.run.smithi077.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-25T05:33:48.013 INFO:teuthology.orchestra.run.smithi077.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-25T05:33:48.013 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:33:48.162 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 1/25 2023-01-25T05:33:48.163 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2/25 2023-01-25T05:33:48.163 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T05:33:48.163 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T05:33:48.164 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 5/25 2023-01-25T05:33:48.164 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 6/25 2023-01-25T05:33:48.164 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T05:33:48.164 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 8/25 2023-01-25T05:33:48.165 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 9/25 2023-01-25T05:33:48.165 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 10/25 2023-01-25T05:33:48.165 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T05:33:48.165 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T05:33:48.166 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 13/25 2023-01-25T05:33:48.166 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-25T05:33:48.166 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-25T05:33:48.166 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-25T05:33:48.167 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-25T05:33:48.167 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-25T05:33:48.167 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-01-25T05:33:48.167 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 20/25 2023-01-25T05:33:48.168 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 21/25 2023-01-25T05:33:48.168 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T05:33:48.168 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T05:33:48.168 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 24/25 2023-01-25T05:33:49.417 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:33:49.528 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T05:33:49.529 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:54 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:33:49.611 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:33:49.611 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2023-01-25T05:33:49.612 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.612 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.612 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:33:49.612 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.613 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.613 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.613 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.613 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T05:33:49.614 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.614 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T05:33:49.614 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T05:33:49.615 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.615 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T05:33:49.615 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.615 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T05:33:49.616 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T05:33:49.616 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.616 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.617 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.617 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T05:33:49.617 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.618 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.618 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:49.618 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T05:33:49.618 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:33:49.619 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:33:49.777 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-test 2023-01-25T05:33:50.121 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:33:50.938 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:55 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:33:52.320 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:33:52.321 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================ 2023-01-25T05:33:52.321 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-25T05:33:52.321 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================ 2023-01-25T05:33:52.321 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:33:52.322 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 47 M 2023-01-25T05:33:52.322 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-25T05:33:52.322 INFO:teuthology.orchestra.run.smithi121.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-01-25T05:33:52.323 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 137 k 2023-01-25T05:33:52.323 INFO:teuthology.orchestra.run.smithi121.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-25T05:33:52.323 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-01-25T05:33:52.323 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-25T05:33:52.323 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:33:52.324 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:33:52.324 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================ 2023-01-25T05:33:52.324 INFO:teuthology.orchestra.run.smithi121.stdout:Install 6 Packages 2023-01-25T05:33:52.324 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:33:52.325 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 48 M 2023-01-25T05:33:52.325 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 218 M 2023-01-25T05:33:52.325 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:33:52.362 INFO:teuthology.orchestra.run.smithi121.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.8 MB/s | 70 kB 00:00 2023-01-25T05:33:52.438 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:33:52.439 INFO:teuthology.orchestra.run.smithi121.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-01-25T05:33:52.488 INFO:teuthology.orchestra.run.smithi121.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-01-25T05:33:52.538 INFO:teuthology.orchestra.run.smithi121.stdout:(4/6): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-01-25T05:33:52.563 INFO:teuthology.orchestra.run.smithi121.stdout:(5/6): libcephsqlite-18.0.0-1959.gb0d25925.el8. 572 kB/s | 137 kB 00:00 2023-01-25T05:33:52.628 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:33:52.628 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2023-01-25T05:33:52.628 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.628 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.629 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:33:52.629 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.629 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.629 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.629 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.630 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T05:33:52.630 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.630 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T05:33:52.630 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T05:33:52.630 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.631 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T05:33:52.631 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.631 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T05:33:52.631 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T05:33:52.631 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.632 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.632 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.632 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T05:33:52.632 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.632 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.633 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:33:52.633 INFO:teuthology.orchestra.run.smithi077.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T05:33:52.633 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:33:52.633 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:33:52.797 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-test 2023-01-25T05:33:53.144 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:33:53.961 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:58 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:33:55.163 INFO:teuthology.orchestra.run.smithi121.stdout:(6/6): ceph-test-18.0.0-1959.gb0d25925.el8.x86_ 17 MB/s | 47 MB 00:02 2023-01-25T05:33:55.165 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:33:55.165 INFO:teuthology.orchestra.run.smithi121.stdout:Total 17 MB/s | 48 MB 00:02 2023-01-25T05:33:55.203 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:33:55.225 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:33:55.225 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:33:55.348 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:33:55.349 INFO:teuthology.orchestra.run.smithi077.stdout:============================================================================================ 2023-01-25T05:33:55.349 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-01-25T05:33:55.349 INFO:teuthology.orchestra.run.smithi077.stdout:============================================================================================ 2023-01-25T05:33:55.349 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:33:55.350 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 47 M 2023-01-25T05:33:55.350 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2023-01-25T05:33:55.350 INFO:teuthology.orchestra.run.smithi077.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-01-25T05:33:55.350 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 137 k 2023-01-25T05:33:55.350 INFO:teuthology.orchestra.run.smithi077.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-25T05:33:55.351 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-01-25T05:33:55.351 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-25T05:33:55.351 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:33:55.351 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:33:55.351 INFO:teuthology.orchestra.run.smithi077.stdout:============================================================================================ 2023-01-25T05:33:55.351 INFO:teuthology.orchestra.run.smithi077.stdout:Install 6 Packages 2023-01-25T05:33:55.352 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:33:55.352 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 48 M 2023-01-25T05:33:55.352 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 218 M 2023-01-25T05:33:55.352 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:33:55.394 INFO:teuthology.orchestra.run.smithi077.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-01-25T05:33:55.460 INFO:teuthology.orchestra.run.smithi077.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-01-25T05:33:55.481 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:33:55.485 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:33:55.502 INFO:teuthology.orchestra.run.smithi077.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-01-25T05:33:55.579 INFO:teuthology.orchestra.run.smithi077.stdout:(4/6): jq-1.6-3.el8.x86_64.rpm 2.6 MB/s | 202 kB 00:00 2023-01-25T05:33:55.628 INFO:teuthology.orchestra.run.smithi077.stdout:(5/6): libcephsqlite-18.0.0-1959.gb0d25925.el8. 494 kB/s | 137 kB 00:00 2023-01-25T05:33:55.956 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:33:56.089 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T05:33:56.174 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T05:33:56.393 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : jq-1.6-3.el8.x86_64 2/6 2023-01-25T05:33:56.548 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2023-01-25T05:33:56.709 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-25T05:33:56.881 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T05:33:58.539 INFO:teuthology.orchestra.run.smithi077.stdout:(6/6): ceph-test-18.0.0-1959.gb0d25925.el8.x86_ 15 MB/s | 47 MB 00:03 2023-01-25T05:33:58.539 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:33:58.539 INFO:teuthology.orchestra.run.smithi077.stdout:Total 15 MB/s | 48 MB 00:03 2023-01-25T05:33:58.576 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:33:58.597 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:33:58.597 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:33:58.855 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:33:58.859 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:33:59.286 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:33:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T05:33:59.499 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T05:33:59.704 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : jq-1.6-3.el8.x86_64 2/6 2023-01-25T05:33:59.875 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2023-01-25T05:34:00.020 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-25T05:34:00.224 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T05:34:01.474 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T05:34:01.735 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T05:34:01.943 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T05:34:01.943 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 1/6 2023-01-25T05:34:01.943 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2/6 2023-01-25T05:34:01.944 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-25T05:34:01.944 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2023-01-25T05:34:01.944 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-25T05:34:03.058 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : jq-1.6-3.el8.x86_64 6/6 2023-01-25T05:34:03.058 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:08 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:04.845 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T05:34:05.087 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T05:34:05.317 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T05:34:05.317 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 1/6 2023-01-25T05:34:05.317 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2/6 2023-01-25T05:34:05.318 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-25T05:34:05.318 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2023-01-25T05:34:05.318 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-25T05:34:05.676 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:34:05.853 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:05.853 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:34:05.854 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:05.854 INFO:teuthology.orchestra.run.smithi121.stdout: jq-1.6-3.el8.x86_64 2023-01-25T05:34:05.854 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:05.854 INFO:teuthology.orchestra.run.smithi121.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T05:34:05.854 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T05:34:05.855 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T05:34:05.855 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:05.855 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:34:05.985 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph 2023-01-25T05:34:06.318 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:06.392 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : jq-1.6-3.el8.x86_64 6/6 2023-01-25T05:34:06.392 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:11 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:07.172 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:12 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:08.635 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:34:08.638 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================= 2023-01-25T05:34:08.639 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-25T05:34:08.639 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================= 2023-01-25T05:34:08.639 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:34:08.639 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.4 k 2023-01-25T05:34:08.639 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-25T05:34:08.640 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 2.3 M 2023-01-25T05:34:08.640 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 1.5 M 2023-01-25T05:34:08.640 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 236 k 2023-01-25T05:34:08.640 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.9 M 2023-01-25T05:34:08.641 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 17 M 2023-01-25T05:34:08.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T05:34:08.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T05:34:08.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-01-25T05:34:08.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T05:34:08.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T05:34:08.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-01-25T05:34:08.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T05:34:08.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T05:34:08.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T05:34:08.643 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-25T05:34:08.643 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-25T05:34:08.643 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T05:34:08.643 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T05:34:08.644 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T05:34:08.644 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-01-25T05:34:08.644 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-01-25T05:34:08.644 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-25T05:34:08.644 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T05:34:08.645 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T05:34:08.645 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T05:34:08.645 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T05:34:08.645 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T05:34:08.645 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T05:34:08.646 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T05:34:08.646 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-25T05:34:08.646 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T05:34:08.646 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:08.646 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:34:08.647 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================= 2023-01-25T05:34:08.647 INFO:teuthology.orchestra.run.smithi121.stdout:Install 32 Packages 2023-01-25T05:34:08.647 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:08.648 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 28 M 2023-01-25T05:34:08.648 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 110 M 2023-01-25T05:34:08.648 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:34:08.848 INFO:teuthology.orchestra.run.smithi121.stdout:(1/32): ceph-18.0.0-1959.gb0d25925.el8.x86_64.r 31 kB/s | 6.4 kB 00:00 2023-01-25T05:34:09.117 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:34:09.297 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:09.298 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:34:09.298 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:09.298 INFO:teuthology.orchestra.run.smithi077.stdout: jq-1.6-3.el8.x86_64 2023-01-25T05:34:09.298 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:09.298 INFO:teuthology.orchestra.run.smithi077.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T05:34:09.299 INFO:teuthology.orchestra.run.smithi077.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T05:34:09.299 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T05:34:09.299 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:09.299 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:34:09.430 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph 2023-01-25T05:34:09.464 INFO:teuthology.orchestra.run.smithi121.stdout:(2/32): ceph-mgr-18.0.0-1959.gb0d25925.el8.x86_ 1.9 MB/s | 1.5 MB 00:00 2023-01-25T05:34:09.590 INFO:teuthology.orchestra.run.smithi121.stdout:(3/32): ceph-mds-18.0.0-1959.gb0d25925.el8.x86_ 2.4 MB/s | 2.3 MB 00:00 2023-01-25T05:34:09.690 INFO:teuthology.orchestra.run.smithi121.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1959.gb0d2 2.3 MB/s | 236 kB 00:00 2023-01-25T05:34:09.762 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:09.824 INFO:teuthology.orchestra.run.smithi121.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 327 kB/s | 44 kB 00:00 2023-01-25T05:34:09.932 INFO:teuthology.orchestra.run.smithi121.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.7 MB/s | 185 kB 00:00 2023-01-25T05:34:10.024 INFO:teuthology.orchestra.run.smithi121.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.8 MB/s | 173 kB 00:00 2023-01-25T05:34:10.125 INFO:teuthology.orchestra.run.smithi121.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 3.8 MB/s | 384 kB 00:00 2023-01-25T05:34:10.175 INFO:teuthology.orchestra.run.smithi121.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 223 kB/s | 11 kB 00:00 2023-01-25T05:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout:(10/32): python3-jaraco-functools-2.0-4.el8.noa 350 kB/s | 18 kB 00:00 2023-01-25T05:34:10.275 INFO:teuthology.orchestra.run.smithi121.stdout:(11/32): python3-logutils-0.3.5-11.el8.noarch.r 978 kB/s | 49 kB 00:00 2023-01-25T05:34:10.326 INFO:teuthology.orchestra.run.smithi121.stdout:(12/32): python3-more-itertools-7.2.0-3.el8.noa 1.1 MB/s | 59 kB 00:00 2023-01-25T05:34:10.414 INFO:teuthology.orchestra.run.smithi121.stdout:(13/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-01-25T05:34:10.460 INFO:teuthology.orchestra.run.smithi121.stdout:(14/32): python3-portend-2.6-1.el8.noarch.rpm 313 kB/s | 16 kB 00:00 2023-01-25T05:34:10.510 INFO:teuthology.orchestra.run.smithi121.stdout:(15/32): python3-simplegeneric-0.8.1-17.el8.noa 373 kB/s | 19 kB 00:00 2023-01-25T05:34:10.577 INFO:teuthology.orchestra.run.smithi121.stdout:(16/32): ceph-mon-18.0.0-1959.gb0d25925.el8.x86 2.3 MB/s | 3.9 MB 00:01 2023-01-25T05:34:10.602 INFO:teuthology.orchestra.run.smithi121.stdout:(17/32): python3-singledispatch-3.4.0.3-18.el8. 260 kB/s | 24 kB 00:00 2023-01-25T05:34:10.626 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:15 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:10.652 INFO:teuthology.orchestra.run.smithi121.stdout:(18/32): python3-trustme-0.6.0-4.el8.noarch.rpm 549 kB/s | 27 kB 00:00 2023-01-25T05:34:10.686 INFO:teuthology.orchestra.run.smithi121.stdout:(19/32): python3-tempora-1.14.1-5.el8.noarch.rp 268 kB/s | 29 kB 00:00 2023-01-25T05:34:10.736 INFO:teuthology.orchestra.run.smithi121.stdout:(20/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.8 MB/s | 241 kB 00:00 2023-01-25T05:34:10.786 INFO:teuthology.orchestra.run.smithi121.stdout:(21/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.7 MB/s | 85 kB 00:00 2023-01-25T05:34:10.812 INFO:teuthology.orchestra.run.smithi121.stdout:(22/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.0 MB/s | 251 kB 00:00 2023-01-25T05:34:10.837 INFO:teuthology.orchestra.run.smithi121.stdout:(23/32): python3-zc-lockfile-2.0-2.el8.noarch.r 456 kB/s | 23 kB 00:00 2023-01-25T05:34:10.887 INFO:teuthology.orchestra.run.smithi121.stdout:(24/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-01-25T05:34:10.912 INFO:teuthology.orchestra.run.smithi121.stdout:(25/32): python3-pycparser-2.14-14.el8.noarch.r 1.5 MB/s | 109 kB 00:00 2023-01-25T05:34:10.946 INFO:teuthology.orchestra.run.smithi121.stdout:(26/32): python3-cryptography-3.2.1-5.el8.x86_6 9.3 MB/s | 559 kB 00:00 2023-01-25T05:34:10.971 INFO:teuthology.orchestra.run.smithi121.stdout:(27/32): python3-pytz-2017.2-9.el8.noarch.rpm 924 kB/s | 54 kB 00:00 2023-01-25T05:34:10.997 INFO:teuthology.orchestra.run.smithi121.stdout:(28/32): python3-markupsafe-0.23-19.el8.x86_64. 771 kB/s | 39 kB 00:00 2023-01-25T05:34:11.022 INFO:teuthology.orchestra.run.smithi121.stdout:(29/32): python3-mako-1.0.6-13.el8.noarch.rpm 3.0 MB/s | 157 kB 00:00 2023-01-25T05:34:11.072 INFO:teuthology.orchestra.run.smithi121.stdout:(30/32): python3-werkzeug-0.12.2-4.el8.noarch.r 5.9 MB/s | 457 kB 00:00 2023-01-25T05:34:11.098 INFO:teuthology.orchestra.run.smithi121.stdout:(31/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 1.3 MB/s | 103 kB 00:00 2023-01-25T05:34:11.993 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:34:11.996 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================================= 2023-01-25T05:34:11.996 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-01-25T05:34:11.996 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================================= 2023-01-25T05:34:11.996 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:34:11.997 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.4 k 2023-01-25T05:34:11.997 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2023-01-25T05:34:11.997 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 2.3 M 2023-01-25T05:34:11.997 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 1.5 M 2023-01-25T05:34:11.998 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 236 k 2023-01-25T05:34:11.998 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.9 M 2023-01-25T05:34:11.998 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 17 M 2023-01-25T05:34:11.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T05:34:11.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T05:34:11.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-01-25T05:34:11.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T05:34:11.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T05:34:11.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-01-25T05:34:11.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T05:34:12.000 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T05:34:12.000 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T05:34:12.000 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-25T05:34:12.000 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-25T05:34:12.000 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T05:34:12.001 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T05:34:12.001 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T05:34:12.001 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-01-25T05:34:12.001 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-01-25T05:34:12.001 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-25T05:34:12.002 INFO:teuthology.orchestra.run.smithi077.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T05:34:12.002 INFO:teuthology.orchestra.run.smithi077.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T05:34:12.002 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T05:34:12.002 INFO:teuthology.orchestra.run.smithi077.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T05:34:12.002 INFO:teuthology.orchestra.run.smithi077.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T05:34:12.003 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T05:34:12.003 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T05:34:12.003 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-25T05:34:12.003 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T05:34:12.003 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:12.004 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:34:12.004 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================================= 2023-01-25T05:34:12.004 INFO:teuthology.orchestra.run.smithi077.stdout:Install 32 Packages 2023-01-25T05:34:12.004 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:12.005 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 28 M 2023-01-25T05:34:12.005 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 110 M 2023-01-25T05:34:12.005 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:34:12.131 INFO:teuthology.orchestra.run.smithi121.stdout:(32/32): ceph-osd-18.0.0-1959.gb0d25925.el8.x86 6.3 MB/s | 17 MB 00:02 2023-01-25T05:34:12.134 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:34:12.134 INFO:teuthology.orchestra.run.smithi121.stdout:Total 8.1 MB/s | 28 MB 00:03 2023-01-25T05:34:12.224 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:34:12.262 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:34:12.262 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:34:12.271 INFO:teuthology.orchestra.run.smithi077.stdout:(1/32): ceph-18.0.0-1959.gb0d25925.el8.x86_64.r 24 kB/s | 6.4 kB 00:00 2023-01-25T05:34:12.465 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:34:12.469 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:34:13.005 INFO:teuthology.orchestra.run.smithi077.stdout:(2/32): ceph-mgr-18.0.0-1959.gb0d25925.el8.x86_ 1.5 MB/s | 1.5 MB 00:01 2023-01-25T05:34:13.230 INFO:teuthology.orchestra.run.smithi077.stdout:(3/32): ceph-mds-18.0.0-1959.gb0d25925.el8.x86_ 1.8 MB/s | 2.3 MB 00:01 2023-01-25T05:34:13.381 INFO:teuthology.orchestra.run.smithi077.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1959.gb0d2 1.5 MB/s | 236 kB 00:00 2023-01-25T05:34:13.531 INFO:teuthology.orchestra.run.smithi077.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 291 kB/s | 44 kB 00:00 2023-01-25T05:34:13.640 INFO:teuthology.orchestra.run.smithi077.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.7 MB/s | 185 kB 00:00 2023-01-25T05:34:13.723 INFO:teuthology.orchestra.run.smithi077.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 2.0 MB/s | 173 kB 00:00 2023-01-25T05:34:13.840 INFO:teuthology.orchestra.run.smithi077.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 3.2 MB/s | 384 kB 00:00 2023-01-25T05:34:13.890 INFO:teuthology.orchestra.run.smithi077.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2023-01-25T05:34:13.941 INFO:teuthology.orchestra.run.smithi077.stdout:(10/32): python3-jaraco-functools-2.0-4.el8.noa 353 kB/s | 18 kB 00:00 2023-01-25T05:34:13.991 INFO:teuthology.orchestra.run.smithi077.stdout:(11/32): python3-logutils-0.3.5-11.el8.noarch.r 976 kB/s | 49 kB 00:00 2023-01-25T05:34:14.041 INFO:teuthology.orchestra.run.smithi077.stdout:(12/32): python3-more-itertools-7.2.0-3.el8.noa 1.2 MB/s | 59 kB 00:00 2023-01-25T05:34:14.125 INFO:teuthology.orchestra.run.smithi077.stdout:(13/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-01-25T05:34:14.175 INFO:teuthology.orchestra.run.smithi077.stdout:(14/32): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-01-25T05:34:14.224 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:34:14.225 INFO:teuthology.orchestra.run.smithi077.stdout:(15/32): python3-simplegeneric-0.8.1-17.el8.noa 375 kB/s | 19 kB 00:00 2023-01-25T05:34:14.275 INFO:teuthology.orchestra.run.smithi077.stdout:(16/32): python3-singledispatch-3.4.0.3-18.el8. 480 kB/s | 24 kB 00:00 2023-01-25T05:34:14.326 INFO:teuthology.orchestra.run.smithi077.stdout:(17/32): python3-tempora-1.14.1-5.el8.noarch.rp 584 kB/s | 29 kB 00:00 2023-01-25T05:34:14.368 INFO:teuthology.orchestra.run.smithi077.stdout:(18/32): python3-trustme-0.6.0-4.el8.noarch.rpm 661 kB/s | 27 kB 00:00 2023-01-25T05:34:14.451 INFO:teuthology.orchestra.run.smithi077.stdout:(19/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.8 MB/s | 241 kB 00:00 2023-01-25T05:34:14.474 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T05:34:14.535 INFO:teuthology.orchestra.run.smithi077.stdout:(20/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 3.0 MB/s | 251 kB 00:00 2023-01-25T05:34:14.586 INFO:teuthology.orchestra.run.smithi077.stdout:(21/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.7 MB/s | 85 kB 00:00 2023-01-25T05:34:14.636 INFO:teuthology.orchestra.run.smithi077.stdout:(22/32): python3-zc-lockfile-2.0-2.el8.noarch.r 458 kB/s | 23 kB 00:00 2023-01-25T05:34:14.760 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T05:34:14.827 INFO:teuthology.orchestra.run.smithi077.stdout:(23/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2023-01-25T05:34:14.869 INFO:teuthology.orchestra.run.smithi077.stdout:(24/32): python3-pycparser-2.14-14.el8.noarch.r 2.6 MB/s | 109 kB 00:00 2023-01-25T05:34:14.911 INFO:teuthology.orchestra.run.smithi077.stdout:(25/32): python3-cryptography-3.2.1-5.el8.x86_6 13 MB/s | 559 kB 00:00 2023-01-25T05:34:14.945 INFO:teuthology.orchestra.run.smithi077.stdout:(26/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-01-25T05:34:14.955 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 3/32 2023-01-25T05:34:15.012 INFO:teuthology.orchestra.run.smithi077.stdout:(27/32): python3-markupsafe-0.23-19.el8.x86_64. 584 kB/s | 39 kB 00:00 2023-01-25T05:34:15.053 INFO:teuthology.orchestra.run.smithi077.stdout:(28/32): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-01-25T05:34:15.096 INFO:teuthology.orchestra.run.smithi077.stdout:(29/32): python3-werkzeug-0.12.2-4.el8.noarch.r 11 MB/s | 457 kB 00:00 2023-01-25T05:34:15.138 INFO:teuthology.orchestra.run.smithi077.stdout:(30/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 2.4 MB/s | 103 kB 00:00 2023-01-25T05:34:15.170 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/32 2023-01-25T05:34:15.196 INFO:teuthology.orchestra.run.smithi077.stdout:(31/32): ceph-mon-18.0.0-1959.gb0d25925.el8.x86 1.3 MB/s | 3.9 MB 00:02 2023-01-25T05:34:15.421 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/32 2023-01-25T05:34:15.605 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 6/32 2023-01-25T05:34:15.756 INFO:teuthology.orchestra.run.smithi077.stdout:(32/32): ceph-osd-18.0.0-1959.gb0d25925.el8.x86 6.1 MB/s | 17 MB 00:02 2023-01-25T05:34:15.757 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:34:15.758 INFO:teuthology.orchestra.run.smithi077.stdout:Total 7.5 MB/s | 28 MB 00:03 2023-01-25T05:34:15.811 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 7/32 2023-01-25T05:34:15.847 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:34:15.884 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:34:15.884 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:34:16.086 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:34:16.090 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:34:16.181 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 8/32 2023-01-25T05:34:16.504 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 9/32 2023-01-25T05:34:16.668 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 10/32 2023-01-25T05:34:16.877 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 11/32 2023-01-25T05:34:17.093 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 12/32 2023-01-25T05:34:17.184 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:34:17.298 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2023-01-25T05:34:17.376 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T05:34:17.458 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 14/32 2023-01-25T05:34:17.639 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2023-01-25T05:34:17.663 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T05:34:17.786 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2023-01-25T05:34:17.834 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 3/32 2023-01-25T05:34:17.946 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2023-01-25T05:34:18.039 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/32 2023-01-25T05:34:18.082 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2023-01-25T05:34:18.258 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/32 2023-01-25T05:34:18.347 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2023-01-25T05:34:18.433 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 6/32 2023-01-25T05:34:18.563 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2023-01-25T05:34:18.694 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 7/32 2023-01-25T05:34:18.757 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T05:34:19.048 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T05:34:19.058 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 8/32 2023-01-25T05:34:19.317 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 9/32 2023-01-25T05:34:19.480 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 10/32 2023-01-25T05:34:19.500 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T05:34:19.626 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 11/32 2023-01-25T05:34:19.728 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2023-01-25T05:34:19.830 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 12/32 2023-01-25T05:34:20.050 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2023-01-25T05:34:20.052 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2023-01-25T05:34:20.220 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 14/32 2023-01-25T05:34:20.338 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T05:34:20.367 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2023-01-25T05:34:20.498 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2023-01-25T05:34:20.658 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 27/32 2023-01-25T05:34:20.815 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T05:34:21.200 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2023-01-25T05:34:21.319 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2023-01-25T05:34:21.550 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2023-01-25T05:34:21.750 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2023-01-25T05:34:21.919 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T05:34:22.159 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T05:34:22.308 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T05:34:22.434 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T05:34:22.531 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T05:34:22.596 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2023-01-25T05:34:22.899 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2023-01-25T05:34:22.916 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T05:34:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T05:34:23.183 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T05:34:23.297 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T05:34:23.467 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T05:34:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T05:34:23.489 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 27/32 2023-01-25T05:34:23.590 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T05:34:23.678 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T05:34:23.934 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T05:34:23.934 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 1/32 2023-01-25T05:34:23.935 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2/32 2023-01-25T05:34:23.935 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 3/32 2023-01-25T05:34:23.935 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 4/32 2023-01-25T05:34:23.935 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 5/32 2023-01-25T05:34:23.935 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 6/32 2023-01-25T05:34:23.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2023-01-25T05:34:23.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2023-01-25T05:34:23.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2023-01-25T05:34:23.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2023-01-25T05:34:23.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2023-01-25T05:34:23.937 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2023-01-25T05:34:23.937 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2023-01-25T05:34:23.937 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2023-01-25T05:34:23.937 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2023-01-25T05:34:23.937 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2023-01-25T05:34:23.938 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2023-01-25T05:34:23.938 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2023-01-25T05:34:23.938 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2023-01-25T05:34:23.938 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2023-01-25T05:34:23.938 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/32 2023-01-25T05:34:23.939 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2023-01-25T05:34:23.939 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2023-01-25T05:34:23.939 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2023-01-25T05:34:23.939 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 25/32 2023-01-25T05:34:23.939 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 26/32 2023-01-25T05:34:23.940 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 27/32 2023-01-25T05:34:23.941 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/32 2023-01-25T05:34:23.941 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 29/32 2023-01-25T05:34:23.941 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 30/32 2023-01-25T05:34:23.941 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 31/32 2023-01-25T05:34:25.167 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T05:34:25.361 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T05:34:25.410 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 32/32 2023-01-25T05:34:25.411 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:30 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T05:34:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T05:34:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T05:34:26.263 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T05:34:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T05:34:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T05:34:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T05:34:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 1/32 2023-01-25T05:34:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2/32 2023-01-25T05:34:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 3/32 2023-01-25T05:34:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 4/32 2023-01-25T05:34:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 5/32 2023-01-25T05:34:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 6/32 2023-01-25T05:34:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2023-01-25T05:34:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2023-01-25T05:34:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2023-01-25T05:34:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2023-01-25T05:34:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2023-01-25T05:34:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2023-01-25T05:34:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2023-01-25T05:34:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2023-01-25T05:34:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2023-01-25T05:34:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2023-01-25T05:34:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2023-01-25T05:34:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2023-01-25T05:34:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2023-01-25T05:34:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2023-01-25T05:34:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/32 2023-01-25T05:34:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2023-01-25T05:34:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2023-01-25T05:34:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2023-01-25T05:34:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 25/32 2023-01-25T05:34:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 26/32 2023-01-25T05:34:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 27/32 2023-01-25T05:34:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/32 2023-01-25T05:34:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 29/32 2023-01-25T05:34:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 30/32 2023-01-25T05:34:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 31/32 2023-01-25T05:34:27.780 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:34:27.967 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:27.967 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:34:27.967 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:27.968 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:27.968 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:27.968 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:34:27.968 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:27.968 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:27.969 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T05:34:27.969 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T05:34:27.969 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T05:34:27.969 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T05:34:27.969 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T05:34:27.970 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T05:34:27.970 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T05:34:27.970 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T05:34:27.970 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T05:34:27.970 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T05:34:27.971 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T05:34:27.971 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T05:34:27.971 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T05:34:27.971 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T05:34:27.971 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T05:34:27.971 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T05:34:27.972 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T05:34:27.972 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T05:34:27.972 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T05:34:27.973 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T05:34:27.973 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T05:34:27.973 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T05:34:27.973 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T05:34:27.973 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T05:34:27.974 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T05:34:27.974 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T05:34:27.974 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:27.974 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:34:28.107 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-base 2023-01-25T05:34:28.152 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 32/32 2023-01-25T05:34:28.153 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:33 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:28.454 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:29.273 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:34 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:30.552 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:34:30.571 INFO:teuthology.orchestra.run.smithi121.stdout:Package ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:34:30.679 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:34:30.680 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-25T05:34:30.680 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:34:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:30.737 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:34:30.738 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:30.738 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:30.738 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:30.738 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:34:30.739 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:30.739 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:30.739 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T05:34:30.739 INFO:teuthology.orchestra.run.smithi077.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T05:34:30.739 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T05:34:30.740 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T05:34:30.740 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T05:34:30.740 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T05:34:30.740 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T05:34:30.741 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T05:34:30.741 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T05:34:30.741 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T05:34:30.741 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T05:34:30.741 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T05:34:30.742 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T05:34:30.742 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T05:34:30.742 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T05:34:30.742 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T05:34:30.743 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T05:34:30.743 INFO:teuthology.orchestra.run.smithi077.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T05:34:30.743 INFO:teuthology.orchestra.run.smithi077.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T05:34:30.743 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T05:34:30.743 INFO:teuthology.orchestra.run.smithi077.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T05:34:30.744 INFO:teuthology.orchestra.run.smithi077.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T05:34:30.744 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T05:34:30.744 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T05:34:30.744 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T05:34:30.745 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T05:34:30.745 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:30.745 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:34:30.891 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-base 2023-01-25T05:34:31.236 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:31.350 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install cephadm 2023-01-25T05:34:31.697 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:32.043 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:37 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:32.503 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:37 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:33.399 INFO:teuthology.orchestra.run.smithi077.stdout:Package ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:34:33.505 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:34:33.506 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2023-01-25T05:34:33.506 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:34:33.611 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install cephadm 2023-01-25T05:34:33.895 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:34:33.895 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:34:33.896 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-25T05:34:33.896 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:34:33.896 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:34:33.896 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 82 k 2023-01-25T05:34:33.897 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:33.897 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:34:33.897 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:34:33.897 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-25T05:34:33.897 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:33.898 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 82 k 2023-01-25T05:34:33.898 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 374 k 2023-01-25T05:34:33.898 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:34:33.954 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:34.076 INFO:teuthology.orchestra.run.smithi121.stdout:cephadm-18.0.0-1959.gb0d25925.el8.noarch.rpm 457 kB/s | 82 kB 00:00 2023-01-25T05:34:34.077 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:34:34.077 INFO:teuthology.orchestra.run.smithi121.stdout:Total 454 kB/s | 82 kB 00:00 2023-01-25T05:34:34.078 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:34:34.085 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:34:34.086 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:34:34.087 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:34:34.091 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:34:34.130 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:34:34.789 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:39 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:35.719 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:34:35.890 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:34:36.053 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:34:36.159 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:34:36.160 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:34:36.160 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-01-25T05:34:36.160 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:34:36.160 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:34:36.161 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 82 k 2023-01-25T05:34:36.161 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:36.161 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:34:36.161 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:34:36.161 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2023-01-25T05:34:36.162 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:36.162 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 82 k 2023-01-25T05:34:36.162 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 374 k 2023-01-25T05:34:36.162 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:34:36.326 INFO:teuthology.orchestra.run.smithi077.stdout:cephadm-18.0.0-1959.gb0d25925.el8.noarch.rpm 497 kB/s | 82 kB 00:00 2023-01-25T05:34:36.327 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:34:36.328 INFO:teuthology.orchestra.run.smithi077.stdout:Total 493 kB/s | 82 kB 00:00 2023-01-25T05:34:36.328 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:34:36.335 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:34:36.335 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:34:36.337 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:34:36.341 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:34:36.380 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:34:37.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:34:37.098 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:42 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:39.245 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:34:39.371 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:34:39.479 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:34:39.537 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:34:39.658 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:39.659 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:34:39.659 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:34:39.659 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:39.659 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:34:39.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T05:34:40.161 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:40.610 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:34:40.610 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:45 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:40.964 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:45 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:42.363 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:34:42.364 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:34:42.364 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2023-01-25T05:34:42.364 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:34:42.364 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:34:42.364 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 166 k 2023-01-25T05:34:42.365 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:42.365 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:34:42.365 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:34:42.365 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-25T05:34:42.366 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:42.366 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 166 k 2023-01-25T05:34:42.366 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 503 k 2023-01-25T05:34:42.366 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:34:43.156 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:34:43.337 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:43.338 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:34:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:34:43.338 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:43.338 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:34:43.423 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-immutable-object-cache-18.0.0-1959.gb0d259 156 kB/s | 166 kB 00:01 2023-01-25T05:34:43.423 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:34:43.424 INFO:teuthology.orchestra.run.smithi121.stdout:Total 156 kB/s | 166 kB 00:01 2023-01-25T05:34:43.424 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:34:43.437 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:34:43.437 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:34:43.476 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T05:34:43.494 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:34:43.497 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:34:43.610 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:34:43.763 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:34:43.831 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:44.111 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:34:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:49 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:45.143 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:34:45.143 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:50 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:46.045 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:34:46.046 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:34:46.046 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repo Size 2023-01-25T05:34:46.046 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:34:46.046 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:34:46.047 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 166 k 2023-01-25T05:34:46.047 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:46.047 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:34:46.047 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:34:46.048 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2023-01-25T05:34:46.049 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:46.049 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 166 k 2023-01-25T05:34:46.049 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 503 k 2023-01-25T05:34:46.049 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:34:46.249 INFO:teuthology.orchestra.run.smithi077.stdout:ceph-immutable-object-cache-18.0.0-1959.gb0d259 819 kB/s | 166 kB 00:00 2023-01-25T05:34:46.249 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:34:46.250 INFO:teuthology.orchestra.run.smithi077.stdout:Total 814 kB/s | 166 kB 00:00 2023-01-25T05:34:46.250 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:34:46.261 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:34:46.261 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:34:46.311 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:34:46.315 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:34:46.444 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:34:46.586 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:34:46.923 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:34:47.489 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:34:47.669 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:47.669 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:34:47.670 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:47.670 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:47.670 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:34:47.810 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr 2023-01-25T05:34:47.914 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T05:34:47.914 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:52 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:48.155 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:48.987 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:53 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:50.255 INFO:teuthology.orchestra.run.smithi121.stdout:Package ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:34:50.295 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:34:50.362 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:34:50.363 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-25T05:34:50.363 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:34:50.470 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T05:34:50.476 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:50.477 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:34:50.477 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:34:50.477 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:50.477 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:34:50.610 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-mgr 2023-01-25T05:34:50.811 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:50.956 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:51.621 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:56 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:51.768 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:56 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:53.009 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:34:53.010 INFO:teuthology.orchestra.run.smithi121.stdout:=================================================================================================== 2023-01-25T05:34:53.010 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-25T05:34:53.010 INFO:teuthology.orchestra.run.smithi121.stdout:=================================================================================================== 2023-01-25T05:34:53.011 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:34:53.011 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 1.7 M 2023-01-25T05:34:53.011 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-25T05:34:53.011 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 23 k 2023-01-25T05:34:53.011 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 15 k 2023-01-25T05:34:53.012 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T05:34:53.012 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T05:34:53.012 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T05:34:53.012 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:53.012 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:34:53.013 INFO:teuthology.orchestra.run.smithi121.stdout:=================================================================================================== 2023-01-25T05:34:53.013 INFO:teuthology.orchestra.run.smithi121.stdout:Install 6 Packages 2023-01-25T05:34:53.013 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:53.014 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 2.0 M 2023-01-25T05:34:53.014 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 8.2 M 2023-01-25T05:34:53.014 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:34:53.062 INFO:teuthology.orchestra.run.smithi077.stdout:Package ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:34:53.170 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:34:53.171 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2023-01-25T05:34:53.172 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:34:53.178 INFO:teuthology.orchestra.run.smithi121.stdout:(1/6): ceph-prometheus-alerts-18.0.0-1959.gb0d2 88 kB/s | 15 kB 00:00 2023-01-25T05:34:53.195 INFO:teuthology.orchestra.run.smithi121.stdout:(2/6): ceph-grafana-dashboards-18.0.0-1959.gb0d 126 kB/s | 23 kB 00:00 2023-01-25T05:34:53.287 INFO:teuthology.orchestra.run.smithi121.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 310 kB/s | 34 kB 00:00 2023-01-25T05:34:53.292 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T05:34:53.328 INFO:teuthology.orchestra.run.smithi121.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-01-25T05:34:53.362 INFO:teuthology.orchestra.run.smithi121.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.1 MB/s | 196 kB 00:00 2023-01-25T05:34:53.462 INFO:teuthology.orchestra.run.smithi121.stdout:(6/6): ceph-mgr-dashboard-18.0.0-1959.gb0d25925 3.8 MB/s | 1.7 MB 00:00 2023-01-25T05:34:53.464 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:34:53.464 INFO:teuthology.orchestra.run.smithi121.stdout:Total 4.5 MB/s | 2.0 MB 00:00 2023-01-25T05:34:53.475 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:34:53.484 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:34:53.485 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:34:53.540 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:34:53.544 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:34:53.632 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:34:53.822 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:34:53.994 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-25T05:34:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-25T05:34:54.363 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-25T05:34:54.447 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:59 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:54.524 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 4/6 2023-01-25T05:34:54.857 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 5/6 2023-01-25T05:34:55.091 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T05:34:55.348 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T05:34:55.348 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 1/6 2023-01-25T05:34:55.349 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 2/6 2023-01-25T05:34:55.349 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 3/6 2023-01-25T05:34:55.349 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-25T05:34:55.349 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-25T05:34:55.824 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:34:55.825 INFO:teuthology.orchestra.run.smithi077.stdout:=================================================================================================== 2023-01-25T05:34:55.825 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-01-25T05:34:55.825 INFO:teuthology.orchestra.run.smithi077.stdout:=================================================================================================== 2023-01-25T05:34:55.825 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:34:55.826 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 1.7 M 2023-01-25T05:34:55.826 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2023-01-25T05:34:55.826 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 23 k 2023-01-25T05:34:55.826 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 15 k 2023-01-25T05:34:55.826 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T05:34:55.827 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T05:34:55.827 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T05:34:55.827 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:55.827 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:34:55.827 INFO:teuthology.orchestra.run.smithi077.stdout:=================================================================================================== 2023-01-25T05:34:55.828 INFO:teuthology.orchestra.run.smithi077.stdout:Install 6 Packages 2023-01-25T05:34:55.828 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:34:55.829 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 2.0 M 2023-01-25T05:34:55.829 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 8.2 M 2023-01-25T05:34:55.829 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:34:55.987 INFO:teuthology.orchestra.run.smithi077.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1959.gb0d 144 kB/s | 23 kB 00:00 2023-01-25T05:34:56.013 INFO:teuthology.orchestra.run.smithi077.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1959.gb0d2 79 kB/s | 15 kB 00:00 2023-01-25T05:34:56.113 INFO:teuthology.orchestra.run.smithi077.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 269 kB/s | 34 kB 00:00 2023-01-25T05:34:56.163 INFO:teuthology.orchestra.run.smithi077.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 851 kB/s | 43 kB 00:00 2023-01-25T05:34:56.188 INFO:teuthology.orchestra.run.smithi077.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.1 MB/s | 196 kB 00:00 2023-01-25T05:34:56.305 INFO:teuthology.orchestra.run.smithi077.stdout:(6/6): ceph-mgr-dashboard-18.0.0-1959.gb0d25925 3.6 MB/s | 1.7 MB 00:00 2023-01-25T05:34:56.307 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:34:56.307 INFO:teuthology.orchestra.run.smithi077.stdout:Total 4.2 MB/s | 2.0 MB 00:00 2023-01-25T05:34:56.318 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:34:56.326 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:34:56.327 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:34:56.355 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-25T05:34:56.356 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:01 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:56.382 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:34:56.385 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:34:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:34:56.818 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-25T05:34:56.981 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-25T05:34:57.136 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-25T05:34:57.273 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 4/6 2023-01-25T05:34:57.587 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 5/6 2023-01-25T05:34:57.826 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T05:34:58.057 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T05:34:58.057 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 1/6 2023-01-25T05:34:58.058 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 2/6 2023-01-25T05:34:58.058 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 3/6 2023-01-25T05:34:58.058 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-25T05:34:58.060 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-25T05:34:58.654 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:34:58.834 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:58.835 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:34:58.835 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:34:58.835 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:34:58.835 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:34:58.836 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T05:34:58.836 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T05:34:58.836 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T05:34:58.836 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:34:58.836 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:34:58.982 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T05:34:59.063 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-25T05:34:59.063 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:04 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:34:59.327 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:00.112 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:05 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:01.566 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:35:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:35:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================================= 2023-01-25T05:35:01.570 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-25T05:35:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================================= 2023-01-25T05:35:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:35:01.571 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 7.4 M 2023-01-25T05:35:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-25T05:35:01.571 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-01-25T05:35:01.572 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-01-25T05:35:01.572 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-25T05:35:01.572 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-01-25T05:35:01.572 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-25T05:35:01.572 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-25T05:35:01.573 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-01-25T05:35:01.573 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:35:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================================= 2023-01-25T05:35:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:Install 8 Packages 2023-01-25T05:35:01.574 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 35 M 2023-01-25T05:35:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 210 M 2023-01-25T05:35:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:35:01.643 INFO:teuthology.orchestra.run.smithi121.stdout:(1/8): libgfortran-8.5.0-10.1.el8_6.x86_64.rpm 8.9 MB/s | 644 kB 00:00 2023-01-25T05:35:01.668 INFO:teuthology.orchestra.run.smithi121.stdout:(2/8): libquadmath-8.5.0-10.1.el8_6.x86_64.rpm 1.7 MB/s | 171 kB 00:00 2023-01-25T05:35:01.748 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:01.748 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:35:01.748 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:35:01.749 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:35:01.749 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:35:01.749 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T05:35:01.749 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T05:35:01.749 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T05:35:01.750 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:01.750 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:35:01.785 INFO:teuthology.orchestra.run.smithi121.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 32 MB/s | 3.7 MB 00:00 2023-01-25T05:35:01.827 INFO:teuthology.orchestra.run.smithi121.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 5.3 MB/s | 225 kB 00:00 2023-01-25T05:35:01.896 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T05:35:01.961 INFO:teuthology.orchestra.run.smithi121.stdout:(5/8): openblas-threads-0.3.15-3.el8.x86_64.rpm 36 MB/s | 4.9 MB 00:00 2023-01-25T05:35:02.102 INFO:teuthology.orchestra.run.smithi121.stdout:(6/8): python3-scipy-1.0.0-21.module+el8.5.0+10 30 MB/s | 14 MB 00:00 2023-01-25T05:35:02.228 INFO:teuthology.orchestra.run.smithi121.stdout:(7/8): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-01-25T05:35:02.240 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:02.786 INFO:teuthology.orchestra.run.smithi121.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-195 6.1 MB/s | 7.4 MB 00:01 2023-01-25T05:35:02.787 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:02.787 INFO:teuthology.orchestra.run.smithi121.stdout:Total 29 MB/s | 35 MB 00:01 2023-01-25T05:35:03.025 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:35:03.043 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:08 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:03.044 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:35:03.044 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:35:03.335 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:35:03.339 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:35:03.884 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:35:04.024 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T05:35:04.151 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T05:35:04.333 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:35:04.411 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:35:04.412 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================================================= 2023-01-25T05:35:04.413 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-01-25T05:35:04.413 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================================================= 2023-01-25T05:35:04.413 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:35:04.414 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 7.4 M 2023-01-25T05:35:04.414 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2023-01-25T05:35:04.414 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-01-25T05:35:04.415 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-01-25T05:35:04.415 INFO:teuthology.orchestra.run.smithi077.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-25T05:35:04.415 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-01-25T05:35:04.416 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-25T05:35:04.416 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-25T05:35:04.416 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-01-25T05:35:04.417 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:04.417 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:35:04.417 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================================================= 2023-01-25T05:35:04.418 INFO:teuthology.orchestra.run.smithi077.stdout:Install 8 Packages 2023-01-25T05:35:04.418 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:04.419 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 35 M 2023-01-25T05:35:04.420 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 210 M 2023-01-25T05:35:04.420 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:35:04.489 INFO:teuthology.orchestra.run.smithi077.stdout:(1/8): libgfortran-8.5.0-10.1.el8_6.x86_64.rpm 8.6 MB/s | 644 kB 00:00 2023-01-25T05:35:04.523 INFO:teuthology.orchestra.run.smithi077.stdout:(2/8): libquadmath-8.5.0-10.1.el8_6.x86_64.rpm 1.6 MB/s | 171 kB 00:00 2023-01-25T05:35:04.631 INFO:teuthology.orchestra.run.smithi077.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 35 MB/s | 3.7 MB 00:00 2023-01-25T05:35:04.673 INFO:teuthology.orchestra.run.smithi077.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 5.3 MB/s | 225 kB 00:00 2023-01-25T05:35:04.807 INFO:teuthology.orchestra.run.smithi077.stdout:(5/8): openblas-threads-0.3.15-3.el8.x86_64.rpm 36 MB/s | 4.9 MB 00:00 2023-01-25T05:35:04.935 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:35:04.949 INFO:teuthology.orchestra.run.smithi077.stdout:(6/8): python3-scipy-1.0.0-21.module+el8.5.0+10 30 MB/s | 14 MB 00:00 2023-01-25T05:35:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:(7/8): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-01-25T05:35:05.141 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T05:35:05.349 INFO:teuthology.orchestra.run.smithi077.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-195 7.9 MB/s | 7.4 MB 00:00 2023-01-25T05:35:05.350 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:05.350 INFO:teuthology.orchestra.run.smithi077.stdout:Total 38 MB/s | 35 MB 00:00 2023-01-25T05:35:05.587 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:35:05.605 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:35:05.606 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:35:05.733 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T05:35:05.896 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:35:05.900 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:35:05.907 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T05:35:06.443 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:35:06.566 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T05:35:06.591 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T05:35:06.703 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T05:35:06.841 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:35:06.885 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T05:35:07.065 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:35:07.440 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:35:07.633 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T05:35:08.222 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T05:35:08.382 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T05:35:08.848 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:35:09.063 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T05:35:09.343 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T05:35:09.516 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:35:10.072 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 7/8 2023-01-25T05:35:10.240 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T05:35:10.489 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T05:35:10.489 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 1/8 2023-01-25T05:35:10.490 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:35:10.490 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 3/8 2023-01-25T05:35:10.490 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 4/8 2023-01-25T05:35:10.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T05:35:10.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:35:10.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 7/8 2023-01-25T05:35:11.261 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:35:11.535 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 8/8 2023-01-25T05:35:11.535 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:16 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:12.504 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 7/8 2023-01-25T05:35:12.665 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T05:35:12.916 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T05:35:12.916 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 1/8 2023-01-25T05:35:12.916 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T05:35:12.917 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 3/8 2023-01-25T05:35:12.917 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 4/8 2023-01-25T05:35:12.917 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T05:35:12.917 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T05:35:12.917 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 7/8 2023-01-25T05:35:13.945 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 8/8 2023-01-25T05:35:13.945 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:18 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:13.988 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:35:14.174 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:14.174 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:35:14.175 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:35:14.175 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-01-25T05:35:14.178 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-01-25T05:35:14.179 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.15-3.el8.x86_64 2023-01-25T05:35:14.179 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-01-25T05:35:14.179 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T05:35:14.179 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T05:35:14.180 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-01-25T05:35:14.180 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:14.180 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:35:14.305 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-rook 2023-01-25T05:35:14.648 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:15.429 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:20 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:16.313 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:35:16.495 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:16.495 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:35:16.496 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:35:16.496 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-01-25T05:35:16.497 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-01-25T05:35:16.497 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-0.3.15-3.el8.x86_64 2023-01-25T05:35:16.497 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-01-25T05:35:16.498 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T05:35:16.498 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T05:35:16.499 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-01-25T05:35:16.499 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:16.499 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:35:16.626 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-mgr-rook 2023-01-25T05:35:16.826 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:35:16.828 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================== 2023-01-25T05:35:16.828 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-25T05:35:16.828 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================== 2023-01-25T05:35:16.828 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:35:16.829 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 47 k 2023-01-25T05:35:16.829 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-25T05:35:16.829 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T05:35:16.829 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T05:35:16.829 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T05:35:16.830 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-25T05:35:16.830 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-25T05:35:16.830 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T05:35:16.830 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-25T05:35:16.830 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-25T05:35:16.831 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-25T05:35:16.831 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T05:35:16.831 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T05:35:16.832 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T05:35:16.832 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:16.832 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:35:16.832 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================== 2023-01-25T05:35:16.832 INFO:teuthology.orchestra.run.smithi121.stdout:Install 13 Packages 2023-01-25T05:35:16.833 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:16.833 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 1.9 M 2023-01-25T05:35:16.833 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 25 M 2023-01-25T05:35:16.834 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:35:16.970 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:16.977 INFO:teuthology.orchestra.run.smithi121.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 111 kB/s | 16 kB 00:00 2023-01-25T05:35:17.008 INFO:teuthology.orchestra.run.smithi121.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 186 kB/s | 33 kB 00:00 2023-01-25T05:35:17.033 INFO:teuthology.orchestra.run.smithi121.stdout:(3/13): ceph-mgr-rook-18.0.0-1959.gb0d25925.el8 234 kB/s | 47 kB 00:00 2023-01-25T05:35:17.083 INFO:teuthology.orchestra.run.smithi121.stdout:(4/13): python3-google-auth-1.1.1-10.el8.noarch 769 kB/s | 81 kB 00:00 2023-01-25T05:35:17.134 INFO:teuthology.orchestra.run.smithi121.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 1.2 MB/s | 61 kB 00:00 2023-01-25T05:35:17.183 INFO:teuthology.orchestra.run.smithi121.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.0 MB/s | 155 kB 00:00 2023-01-25T05:35:17.209 INFO:teuthology.orchestra.run.smithi121.stdout:(7/13): python3-rsa-4.8-1.el8.noarch.rpm 350 kB/s | 61 kB 00:00 2023-01-25T05:35:17.234 INFO:teuthology.orchestra.run.smithi121.stdout:(8/13): python3-requests-oauthlib-1.0.0-1.el8.n 848 kB/s | 43 kB 00:00 2023-01-25T05:35:17.276 INFO:teuthology.orchestra.run.smithi121.stdout:(9/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 731 kB/s | 27 kB 00:00 2023-01-25T05:35:17.301 INFO:teuthology.orchestra.run.smithi121.stdout:(10/13): python3-pyasn1-modules-0.3.7-6.el8.noa 1.2 MB/s | 110 kB 00:00 2023-01-25T05:35:17.327 INFO:teuthology.orchestra.run.smithi121.stdout:(11/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.4 MB/s | 126 kB 00:00 2023-01-25T05:35:17.352 INFO:teuthology.orchestra.run.smithi121.stdout:(12/13): python3-jsonpointer-1.10-11.el8.noarch 398 kB/s | 20 kB 00:00 2023-01-25T05:35:17.385 INFO:teuthology.orchestra.run.smithi121.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.1 MB/s | 1.2 MB 00:00 2023-01-25T05:35:17.386 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:17.386 INFO:teuthology.orchestra.run.smithi121.stdout:Total 3.5 MB/s | 1.9 MB 00:00 2023-01-25T05:35:17.446 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:35:17.461 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:35:17.461 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:35:17.531 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:35:17.535 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:35:17.832 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:22 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:18.029 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:35:18.230 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T05:35:18.473 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-25T05:35:18.644 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-25T05:35:18.786 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-25T05:35:18.981 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-25T05:35:19.218 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:35:19.219 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================================== 2023-01-25T05:35:19.220 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-01-25T05:35:19.220 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================================== 2023-01-25T05:35:19.220 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:35:19.220 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 47 k 2023-01-25T05:35:19.220 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2023-01-25T05:35:19.221 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T05:35:19.221 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T05:35:19.221 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T05:35:19.221 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-25T05:35:19.222 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-25T05:35:19.222 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T05:35:19.222 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-25T05:35:19.222 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-25T05:35:19.222 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-25T05:35:19.223 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T05:35:19.223 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T05:35:19.223 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T05:35:19.223 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:19.223 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:35:19.224 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================================== 2023-01-25T05:35:19.224 INFO:teuthology.orchestra.run.smithi077.stdout:Install 13 Packages 2023-01-25T05:35:19.224 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:19.225 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 1.9 M 2023-01-25T05:35:19.225 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 25 M 2023-01-25T05:35:19.225 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:35:19.258 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T05:35:19.348 INFO:teuthology.orchestra.run.smithi077.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 131 kB/s | 16 kB 00:00 2023-01-25T05:35:19.373 INFO:teuthology.orchestra.run.smithi077.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 220 kB/s | 33 kB 00:00 2023-01-25T05:35:19.398 INFO:teuthology.orchestra.run.smithi077.stdout:(3/13): ceph-mgr-rook-18.0.0-1959.gb0d25925.el8 269 kB/s | 47 kB 00:00 2023-01-25T05:35:19.423 INFO:teuthology.orchestra.run.smithi077.stdout:(4/13): python3-google-auth-1.1.1-10.el8.noarch 1.0 MB/s | 81 kB 00:00 2023-01-25T05:35:19.464 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T05:35:19.474 INFO:teuthology.orchestra.run.smithi077.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 1.2 MB/s | 61 kB 00:00 2023-01-25T05:35:19.533 INFO:teuthology.orchestra.run.smithi077.stdout:(6/13): python3-rsa-4.8-1.el8.noarch.rpm 459 kB/s | 61 kB 00:00 2023-01-25T05:35:19.558 INFO:teuthology.orchestra.run.smithi077.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.8 MB/s | 155 kB 00:00 2023-01-25T05:35:19.600 INFO:teuthology.orchestra.run.smithi077.stdout:(8/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.9 MB/s | 110 kB 00:00 2023-01-25T05:35:19.652 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-25T05:35:19.655 INFO:teuthology.orchestra.run.smithi077.stdout:(9/13): python3-requests-oauthlib-1.0.0-1.el8.n 348 kB/s | 43 kB 00:00 2023-01-25T05:35:19.684 INFO:teuthology.orchestra.run.smithi077.stdout:(10/13): python3-jsonpatch-1.21-2.el8.noarch.rp 317 kB/s | 27 kB 00:00 2023-01-25T05:35:19.726 INFO:teuthology.orchestra.run.smithi077.stdout:(11/13): python3-kubernetes-11.0.0-6.el8.noarch 3.3 MB/s | 1.2 MB 00:00 2023-01-25T05:35:19.751 INFO:teuthology.orchestra.run.smithi077.stdout:(12/13): python3-jsonpointer-1.10-11.el8.noarch 300 kB/s | 20 kB 00:00 2023-01-25T05:35:19.777 INFO:teuthology.orchestra.run.smithi077.stdout:(13/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2023-01-25T05:35:19.777 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:19.778 INFO:teuthology.orchestra.run.smithi077.stdout:Total 3.5 MB/s | 1.9 MB 00:00 2023-01-25T05:35:19.826 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T05:35:19.836 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:35:19.851 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:35:19.851 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:35:19.920 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:35:19.924 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:35:19.999 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-25T05:35:20.392 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:35:20.572 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T05:35:20.757 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-25T05:35:20.929 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-25T05:35:21.051 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-25T05:35:21.088 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-25T05:35:21.279 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-25T05:35:21.282 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-25T05:35:21.412 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T05:35:21.510 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T05:35:21.646 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T05:35:21.647 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 1/13 2023-01-25T05:35:21.647 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-25T05:35:21.647 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-25T05:35:21.647 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-25T05:35:21.647 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-25T05:35:21.648 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-25T05:35:21.648 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-25T05:35:21.648 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T05:35:21.648 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T05:35:21.648 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-01-25T05:35:21.649 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 11/13 2023-01-25T05:35:21.649 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 12/13 2023-01-25T05:35:21.699 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T05:35:21.871 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-25T05:35:22.019 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T05:35:22.216 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-25T05:35:22.715 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-25T05:35:22.715 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:27 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:23.290 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-25T05:35:23.505 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-25T05:35:23.639 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T05:35:23.862 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T05:35:23.863 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 1/13 2023-01-25T05:35:23.863 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-25T05:35:23.863 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-25T05:35:23.863 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-25T05:35:23.863 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-25T05:35:23.864 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-25T05:35:23.864 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-25T05:35:23.864 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T05:35:23.864 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T05:35:23.864 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-01-25T05:35:23.865 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 11/13 2023-01-25T05:35:23.865 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 12/13 2023-01-25T05:35:24.931 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-25T05:35:24.931 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:29 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:25.100 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:35:25.281 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:25.281 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:35:25.281 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:35:25.282 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T05:35:25.282 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T05:35:25.282 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T05:35:25.282 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T05:35:25.283 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T05:35:25.283 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T05:35:25.283 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T05:35:25.284 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T05:35:25.284 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T05:35:25.284 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T05:35:25.284 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T05:35:25.285 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T05:35:25.285 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:25.285 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:35:25.409 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T05:35:25.758 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:26.539 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:31 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:27.335 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:35:27.518 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:27.518 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:35:27.518 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:35:27.519 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T05:35:27.519 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T05:35:27.519 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T05:35:27.519 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T05:35:27.519 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T05:35:27.520 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T05:35:27.520 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T05:35:27.520 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T05:35:27.520 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T05:35:27.520 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T05:35:27.521 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T05:35:27.521 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T05:35:27.521 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:27.521 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:35:27.658 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T05:35:27.928 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:35:27.928 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================================== 2023-01-25T05:35:27.929 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-25T05:35:27.929 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================================== 2023-01-25T05:35:27.929 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:35:27.929 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 120 k 2023-01-25T05:35:27.929 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-25T05:35:27.930 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T05:35:27.930 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-25T05:35:27.930 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-25T05:35:27.930 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T05:35:27.931 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:27.931 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:35:27.931 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================================== 2023-01-25T05:35:27.931 INFO:teuthology.orchestra.run.smithi121.stdout:Install 5 Packages 2023-01-25T05:35:27.932 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:27.933 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 6.0 M 2023-01-25T05:35:27.933 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 26 M 2023-01-25T05:35:27.934 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:35:28.007 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:28.075 INFO:teuthology.orchestra.run.smithi121.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 418 kB/s | 60 kB 00:00 2023-01-25T05:35:28.117 INFO:teuthology.orchestra.run.smithi121.stdout:(2/5): ceph-mgr-cephadm-18.0.0-1959.gb0d25925.e 644 kB/s | 120 kB 00:00 2023-01-25T05:35:28.152 INFO:teuthology.orchestra.run.smithi121.stdout:(3/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 7.0 MB/s | 538 kB 00:00 2023-01-25T05:35:28.184 INFO:teuthology.orchestra.run.smithi121.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.8 MB/s | 476 kB 00:00 2023-01-25T05:35:28.293 INFO:teuthology.orchestra.run.smithi121.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 27 MB/s | 4.8 MB 00:00 2023-01-25T05:35:28.294 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:28.294 INFO:teuthology.orchestra.run.smithi121.stdout:Total 16 MB/s | 6.0 MB 00:00 2023-01-25T05:35:28.346 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:35:28.357 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:35:28.357 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:35:28.435 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:35:28.438 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:35:28.828 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:33 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:29.402 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:35:29.733 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2023-01-25T05:35:29.953 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2023-01-25T05:35:30.283 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T05:35:30.284 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:35:30.285 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================================== 2023-01-25T05:35:30.286 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-01-25T05:35:30.286 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================================== 2023-01-25T05:35:30.286 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:35:30.286 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 120 k 2023-01-25T05:35:30.286 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2023-01-25T05:35:30.287 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T05:35:30.287 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-25T05:35:30.287 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-25T05:35:30.287 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T05:35:30.287 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:30.288 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:35:30.288 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================================== 2023-01-25T05:35:30.288 INFO:teuthology.orchestra.run.smithi077.stdout:Install 5 Packages 2023-01-25T05:35:30.289 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:30.289 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 6.0 M 2023-01-25T05:35:30.290 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 26 M 2023-01-25T05:35:30.290 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:35:30.501 INFO:teuthology.orchestra.run.smithi077.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 283 kB/s | 60 kB 00:00 2023-01-25T05:35:30.576 INFO:teuthology.orchestra.run.smithi077.stdout:(2/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 7.0 MB/s | 538 kB 00:00 2023-01-25T05:35:30.597 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T05:35:30.601 INFO:teuthology.orchestra.run.smithi077.stdout:(3/5): ceph-mgr-cephadm-18.0.0-1959.gb0d25925.e 383 kB/s | 120 kB 00:00 2023-01-25T05:35:30.627 INFO:teuthology.orchestra.run.smithi077.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.4 MB/s | 476 kB 00:00 2023-01-25T05:35:30.721 INFO:teuthology.orchestra.run.smithi077.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 34 MB/s | 4.8 MB 00:00 2023-01-25T05:35:30.721 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:30.724 INFO:teuthology.orchestra.run.smithi077.stdout:Total 14 MB/s | 6.0 MB 00:00 2023-01-25T05:35:30.773 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:35:30.777 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T05:35:30.784 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:35:30.784 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:35:30.862 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:35:30.866 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:35:31.011 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T05:35:31.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/5 2023-01-25T05:35:31.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T05:35:31.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T05:35:31.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T05:35:31.836 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:35:32.039 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 5/5 2023-01-25T05:35:32.040 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:37 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:32.149 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2023-01-25T05:35:32.356 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2023-01-25T05:35:32.666 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T05:35:32.924 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T05:35:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T05:35:33.387 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T05:35:33.388 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/5 2023-01-25T05:35:33.388 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T05:35:33.388 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T05:35:33.388 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T05:35:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 5/5 2023-01-25T05:35:34.418 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:39 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:34.481 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:35:34.664 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:34.664 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:35:34.665 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:35:34.665 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T05:35:34.665 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T05:35:34.665 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T05:35:34.665 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T05:35:34.666 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:34.666 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:35:34.782 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-fuse 2023-01-25T05:35:35.133 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:35.949 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:40 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:36.974 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:35:37.156 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:37.157 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:35:37.157 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:35:37.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T05:35:37.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T05:35:37.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T05:35:37.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T05:35:37.159 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:37.159 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:35:37.276 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-fuse 2023-01-25T05:35:37.323 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:35:37.323 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:35:37.324 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-25T05:35:37.324 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:35:37.324 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:35:37.324 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse 2023-01-25T05:35:37.325 INFO:teuthology.orchestra.run.smithi121.stdout: x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 845 k 2023-01-25T05:35:37.325 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-25T05:35:37.325 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-25T05:35:37.326 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:37.326 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:35:37.327 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:35:37.327 INFO:teuthology.orchestra.run.smithi121.stdout:Install 2 Packages 2023-01-25T05:35:37.327 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:37.329 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 927 k 2023-01-25T05:35:37.329 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 2.9 M 2023-01-25T05:35:37.329 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:35:37.421 INFO:teuthology.orchestra.run.smithi121.stdout:(1/2): fuse-2.9.7-15.el8.x86_64.rpm 860 kB/s | 83 kB 00:00 2023-01-25T05:35:37.627 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:38.126 INFO:teuthology.orchestra.run.smithi121.stdout:(2/2): ceph-fuse-18.0.0-1959.gb0d25925.el8.x86_ 1.0 MB/s | 845 kB 00:00 2023-01-25T05:35:38.127 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:38.128 INFO:teuthology.orchestra.run.smithi121.stdout:Total 1.1 MB/s | 927 kB 00:00 2023-01-25T05:35:38.137 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:35:38.159 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:35:38.160 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:35:38.234 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:35:38.238 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:35:38.404 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:35:38.434 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:43 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:38.648 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.7-15.el8.x86_64 1/2 2023-01-25T05:35:38.824 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T05:35:39.164 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T05:35:39.165 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/2 2023-01-25T05:35:39.813 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:35:39.813 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:35:39.814 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-01-25T05:35:39.814 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:35:39.814 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:35:39.814 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse 2023-01-25T05:35:39.814 INFO:teuthology.orchestra.run.smithi077.stdout: x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 845 k 2023-01-25T05:35:39.815 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2023-01-25T05:35:39.815 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-25T05:35:39.815 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:39.815 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:35:39.815 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:35:39.816 INFO:teuthology.orchestra.run.smithi077.stdout:Install 2 Packages 2023-01-25T05:35:39.816 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:39.816 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 927 k 2023-01-25T05:35:39.817 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 2.9 M 2023-01-25T05:35:39.817 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:35:39.886 INFO:teuthology.orchestra.run.smithi077.stdout:(1/2): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-01-25T05:35:40.219 INFO:teuthology.orchestra.run.smithi077.stdout:(2/2): ceph-fuse-18.0.0-1959.gb0d25925.el8.x86_ 2.0 MB/s | 845 kB 00:00 2023-01-25T05:35:40.220 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:40.220 INFO:teuthology.orchestra.run.smithi077.stdout:Total 2.2 MB/s | 927 kB 00:00 2023-01-25T05:35:40.231 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:35:40.251 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:35:40.252 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:35:40.325 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:35:40.328 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 2/2 2023-01-25T05:35:40.328 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:45 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:40.329 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:35:40.493 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:35:40.760 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.7-15.el8.x86_64 1/2 2023-01-25T05:35:40.910 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T05:35:41.253 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T05:35:41.254 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/2 2023-01-25T05:35:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:35:42.687 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 2/2 2023-01-25T05:35:42.687 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:47 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:42.811 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:42.812 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:35:42.812 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 fuse-2.9.7-15.el8.x86_64 2023-01-25T05:35:42.812 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:42.812 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:35:42.942 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-volume 2023-01-25T05:35:43.298 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:44.077 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:49 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:45.063 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:35:45.242 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:45.243 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:35:45.243 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 fuse-2.9.7-15.el8.x86_64 2023-01-25T05:35:45.243 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:45.243 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:35:45.374 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-volume 2023-01-25T05:35:45.447 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:35:45.448 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:35:45.449 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-25T05:35:45.449 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:35:45.449 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:35:45.449 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 263 k 2023-01-25T05:35:45.450 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:45.450 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:35:45.450 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:35:45.450 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-25T05:35:45.450 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:45.451 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 263 k 2023-01-25T05:35:45.451 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 1.3 M 2023-01-25T05:35:45.451 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:35:45.699 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-volume-18.0.0-1959.gb0d25925.el8.noarch.rp 1.0 MB/s | 263 kB 00:00 2023-01-25T05:35:45.700 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:45.700 INFO:teuthology.orchestra.run.smithi121.stdout:Total 1.0 MB/s | 263 kB 00:00 2023-01-25T05:35:45.701 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:35:45.711 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:35:45.711 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:35:45.727 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:35:45.728 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:45.731 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:35:45.898 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:35:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:35:46.489 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:35:46.560 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:51 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:47.543 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:35:47.543 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:52 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:47.940 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:35:47.941 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:35:47.941 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-01-25T05:35:47.941 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:35:47.941 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:35:47.942 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 263 k 2023-01-25T05:35:47.942 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:47.942 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:35:47.942 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:35:47.943 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2023-01-25T05:35:47.943 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:47.943 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 263 k 2023-01-25T05:35:47.943 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 1.3 M 2023-01-25T05:35:47.943 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:35:48.187 INFO:teuthology.orchestra.run.smithi077.stdout:ceph-volume-18.0.0-1959.gb0d25925.el8.noarch.rp 1.0 MB/s | 263 kB 00:00 2023-01-25T05:35:48.188 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:48.188 INFO:teuthology.orchestra.run.smithi077.stdout:Total 1.0 MB/s | 263 kB 00:00 2023-01-25T05:35:48.188 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:35:48.198 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:35:48.198 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:35:48.215 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:35:48.218 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:35:48.381 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:35:48.644 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:35:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:35:49.873 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:35:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T05:35:50.039 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:55 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:50.053 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:50.053 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:35:50.053 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:35:50.053 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:50.054 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:35:50.190 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librados-devel 2023-01-25T05:35:50.544 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:51.339 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:56 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:52.376 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:35:52.555 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:35:52.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T05:35:52.556 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:52.556 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:35:52.689 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install librados-devel 2023-01-25T05:35:52.717 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:35:52.719 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:35:52.719 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2023-01-25T05:35:52.719 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:35:52.720 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:35:52.720 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 139 k 2023-01-25T05:35:52.720 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:52.720 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:35:52.720 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:35:52.721 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-25T05:35:52.721 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:52.721 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 139 k 2023-01-25T05:35:52.721 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 512 k 2023-01-25T05:35:52.721 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:35:52.922 INFO:teuthology.orchestra.run.smithi121.stdout:librados-devel-18.0.0-1959.gb0d25925.el8.x86_64 683 kB/s | 139 kB 00:00 2023-01-25T05:35:52.923 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:52.924 INFO:teuthology.orchestra.run.smithi121.stdout:Total 679 kB/s | 139 kB 00:00 2023-01-25T05:35:52.924 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:35:52.929 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:35:52.929 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:35:52.982 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:35:52.986 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:35:53.043 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:53.103 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:35:53.253 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:35:53.467 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:35:53.828 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:58 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:54.542 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:35:54.542 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:59 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:55.205 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:35:55.206 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:35:55.207 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repo Size 2023-01-25T05:35:55.207 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:35:55.207 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:35:55.207 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 139 k 2023-01-25T05:35:55.207 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:55.208 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:35:55.208 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:35:55.208 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2023-01-25T05:35:55.208 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:55.208 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 139 k 2023-01-25T05:35:55.209 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 512 k 2023-01-25T05:35:55.209 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:35:55.413 INFO:teuthology.orchestra.run.smithi077.stdout:librados-devel-18.0.0-1959.gb0d25925.el8.x86_64 679 kB/s | 139 kB 00:00 2023-01-25T05:35:55.414 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:35:55.414 INFO:teuthology.orchestra.run.smithi077.stdout:Total 675 kB/s | 139 kB 00:00 2023-01-25T05:35:55.414 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:35:55.419 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:35:55.419 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:35:55.472 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:35:55.476 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:35:55.586 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:35:55.733 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:35:55.968 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:35:56.849 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:35:57.024 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:35:57.025 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:02 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:57.032 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:57.032 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:35:57.032 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:35:57.032 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:35:57.032 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:35:57.153 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install libcephfs2 2023-01-25T05:35:57.507 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:35:58.359 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:03 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:35:59.369 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:35:59.549 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:59.549 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:35:59.549 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:35:59.550 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:35:59.550 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:35:59.659 INFO:teuthology.orchestra.run.smithi121.stdout:Package libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:35:59.669 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install libcephfs2 2023-01-25T05:35:59.764 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:35:59.765 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-25T05:35:59.766 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:35:59.870 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install libcephfs-devel 2023-01-25T05:36:00.023 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:00.224 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:00.914 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:05 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:01.115 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:06 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:02.174 INFO:teuthology.orchestra.run.smithi077.stdout:Package libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:02.283 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:36:02.284 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2023-01-25T05:36:02.284 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:36:02.392 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install libcephfs-devel 2023-01-25T05:36:02.492 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:36:02.493 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:02.493 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2023-01-25T05:36:02.493 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:02.494 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:36:02.494 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 30 k 2023-01-25T05:36:02.494 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:02.494 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:36:02.494 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:02.495 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-25T05:36:02.495 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:02.495 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 30 k 2023-01-25T05:36:02.495 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 137 k 2023-01-25T05:36:02.495 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:36:02.745 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:02.993 INFO:teuthology.orchestra.run.smithi121.stdout:libcephfs-devel-18.0.0-1959.gb0d25925.el8.x86_6 61 kB/s | 30 kB 00:00 2023-01-25T05:36:02.994 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:36:02.999 INFO:teuthology.orchestra.run.smithi121.stdout:Total 60 kB/s | 30 kB 00:00 2023-01-25T05:36:03.000 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:36:03.003 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:36:03.004 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:36:03.015 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:36:03.019 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:36:03.115 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:36:03.292 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:03.502 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:03.515 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:08 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:04.514 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:04.514 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:09 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:04.919 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:36:04.920 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:04.920 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repo Size 2023-01-25T05:36:04.920 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:04.920 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:36:04.921 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 30 k 2023-01-25T05:36:04.921 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:04.921 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:36:04.921 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:04.921 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2023-01-25T05:36:04.922 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:04.922 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 30 k 2023-01-25T05:36:04.922 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 137 k 2023-01-25T05:36:04.922 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:36:05.269 INFO:teuthology.orchestra.run.smithi077.stdout:libcephfs-devel-18.0.0-1959.gb0d25925.el8.x86_6 87 kB/s | 30 kB 00:00 2023-01-25T05:36:05.270 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:36:05.273 INFO:teuthology.orchestra.run.smithi077.stdout:Total 86 kB/s | 30 kB 00:00 2023-01-25T05:36:05.274 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:36:05.278 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:36:05.278 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:36:05.289 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:36:05.293 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:36:05.373 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:36:05.523 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:05.738 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:06.728 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:06.729 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:11 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:06.830 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:36:07.012 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:07.013 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:36:07.013 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:36:07.013 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:07.013 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:36:07.132 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librados2 2023-01-25T05:36:07.487 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:08.273 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:13 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:09.016 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:36:09.198 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:09.198 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:36:09.199 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:36:09.199 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:09.199 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:36:09.336 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install librados2 2023-01-25T05:36:09.544 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:09.649 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:36:09.651 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-25T05:36:09.651 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:36:09.687 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:09.755 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librbd1 2023-01-25T05:36:10.107 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:10.487 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:15 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:10.931 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:15 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:11.749 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:11.857 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:36:11.858 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2023-01-25T05:36:11.858 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:36:11.971 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install librbd1 2023-01-25T05:36:12.285 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:12.321 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:12.397 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:36:12.398 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-25T05:36:12.398 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:36:12.505 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rados 2023-01-25T05:36:12.858 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:13.314 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:18 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:13.670 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:18 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:14.588 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:14.694 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:36:14.695 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2023-01-25T05:36:14.695 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:36:14.816 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-rados 2023-01-25T05:36:15.023 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:15.134 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:36:15.136 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-25T05:36:15.136 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:36:15.166 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:15.243 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rgw 2023-01-25T05:36:15.595 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:15.959 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:20 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:16.483 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:21 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:17.232 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:17.339 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:36:17.340 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2023-01-25T05:36:17.340 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:36:17.456 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-rgw 2023-01-25T05:36:17.743 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:17.807 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:17.853 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:36:17.853 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-25T05:36:17.853 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:36:17.953 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-cephfs 2023-01-25T05:36:18.305 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:18.587 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:23 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:19.090 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:24 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:19.939 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:20.047 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:36:20.048 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2023-01-25T05:36:20.049 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:36:20.174 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-cephfs 2023-01-25T05:36:20.350 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:20.455 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:36:20.457 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-25T05:36:20.457 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:36:20.524 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:20.568 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rbd 2023-01-25T05:36:20.921 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:21.307 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:26 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:21.716 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:26 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:22.576 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:22.682 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:36:22.683 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2023-01-25T05:36:22.684 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:36:22.793 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-rbd 2023-01-25T05:36:23.061 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:23.144 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:23.168 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:36:23.169 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-25T05:36:23.170 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:36:23.269 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-fuse 2023-01-25T05:36:23.620 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:24.044 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:29 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:24.432 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:29 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:25.302 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:36:25.407 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:36:25.409 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2023-01-25T05:36:25.409 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:36:25.637 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install rbd-fuse 2023-01-25T05:36:25.828 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:36:25.829 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:25.829 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2023-01-25T05:36:25.829 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:25.830 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:36:25.830 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 94 k 2023-01-25T05:36:25.830 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:25.830 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:36:25.830 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:25.830 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-25T05:36:25.831 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:25.831 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 94 k 2023-01-25T05:36:25.831 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 280 k 2023-01-25T05:36:25.831 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:36:25.989 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:26.028 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-fuse-18.0.0-1959.gb0d25925.el8.x86_64.rpm 477 kB/s | 94 kB 00:00 2023-01-25T05:36:26.028 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:36:26.029 INFO:teuthology.orchestra.run.smithi121.stdout:Total 474 kB/s | 94 kB 00:00 2023-01-25T05:36:26.029 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:36:26.039 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:36:26.039 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:36:26.092 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:36:26.096 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:36:26.228 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:36:26.387 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:26.605 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:26.770 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:31 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:27.657 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:27.657 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:32 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:28.138 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:36:28.139 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:28.139 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2023-01-25T05:36:28.139 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:28.140 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:36:28.140 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 94 k 2023-01-25T05:36:28.140 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:28.140 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:36:28.140 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:28.141 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2023-01-25T05:36:28.141 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:28.141 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 94 k 2023-01-25T05:36:28.141 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 280 k 2023-01-25T05:36:28.141 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:36:28.313 INFO:teuthology.orchestra.run.smithi077.stdout:rbd-fuse-18.0.0-1959.gb0d25925.el8.x86_64.rpm 545 kB/s | 94 kB 00:00 2023-01-25T05:36:28.314 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:36:28.315 INFO:teuthology.orchestra.run.smithi077.stdout:Total 541 kB/s | 94 kB 00:00 2023-01-25T05:36:28.315 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:36:28.324 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:36:28.324 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:36:28.379 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:36:28.382 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:36:28.522 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:36:28.670 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:28.900 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:29.946 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:29.947 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:34 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:30.018 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:36:30.201 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:30.201 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:36:30.201 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:36:30.202 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:30.202 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:36:30.322 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-mirror 2023-01-25T05:36:30.677 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:31.465 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:36 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:32.251 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:36:32.434 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:32.434 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:36:32.434 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:36:32.435 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:32.435 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:36:32.563 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install rbd-mirror 2023-01-25T05:36:32.872 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:36:32.873 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:32.873 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2023-01-25T05:36:32.874 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:32.874 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:36:32.874 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.4 M 2023-01-25T05:36:32.874 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:32.874 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:36:32.875 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:32.875 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-25T05:36:32.875 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:32.875 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 3.4 M 2023-01-25T05:36:32.876 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 16 M 2023-01-25T05:36:32.876 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:36:32.916 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:33.671 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-mirror-18.0.0-1959.gb0d25925.el8.x86_64.rpm 4.3 MB/s | 3.4 MB 00:00 2023-01-25T05:36:33.672 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:36:33.672 INFO:teuthology.orchestra.run.smithi121.stdout:Total 4.3 MB/s | 3.4 MB 00:00 2023-01-25T05:36:33.673 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:36:33.685 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:36:33.685 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:36:33.752 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:36:33.756 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:36:33.807 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:38 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:34.320 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:36:34.480 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:34.858 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:35.175 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:36:35.176 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:35.176 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repo Size 2023-01-25T05:36:35.176 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:35.177 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:36:35.177 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.4 M 2023-01-25T05:36:35.177 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:35.177 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:36:35.177 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:35.178 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2023-01-25T05:36:35.178 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:35.178 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 3.4 M 2023-01-25T05:36:35.178 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 16 M 2023-01-25T05:36:35.178 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:36:35.825 INFO:teuthology.orchestra.run.smithi077.stdout:rbd-mirror-18.0.0-1959.gb0d25925.el8.x86_64.rpm 5.3 MB/s | 3.4 MB 00:00 2023-01-25T05:36:35.826 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:36:35.826 INFO:teuthology.orchestra.run.smithi077.stdout:Total 5.3 MB/s | 3.4 MB 00:00 2023-01-25T05:36:35.826 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:36:35.838 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:36:35.839 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:36:35.873 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:35.873 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:40 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:35.905 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:36:35.908 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:36:36.349 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:36:36.522 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:36.887 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:37.919 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:37.919 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:42 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:38.334 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:36:38.514 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:38.514 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:36:38.514 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:36:38.515 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:38.515 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:36:38.637 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-nbd 2023-01-25T05:36:38.991 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:39.774 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:44 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:40.331 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:36:40.511 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:40.511 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:36:40.511 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:36:40.512 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:40.512 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:36:40.642 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install rbd-nbd 2023-01-25T05:36:40.996 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-01-25T05:36:41.162 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-25T05:36:41.163 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:41.163 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2023-01-25T05:36:41.164 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:41.164 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-25T05:36:41.164 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 179 k 2023-01-25T05:36:41.164 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:41.165 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-25T05:36:41.165 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-25T05:36:41.165 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-25T05:36:41.165 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:41.165 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 179 k 2023-01-25T05:36:41.166 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 568 k 2023-01-25T05:36:41.166 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-25T05:36:41.359 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-nbd-18.0.0-1959.gb0d25925.el8.x86_64.rpm 920 kB/s | 179 kB 00:00 2023-01-25T05:36:41.360 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:36:41.360 INFO:teuthology.orchestra.run.smithi121.stdout:Total 914 kB/s | 179 kB 00:00 2023-01-25T05:36:41.361 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-25T05:36:41.372 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-25T05:36:41.372 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-25T05:36:41.426 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-25T05:36:41.429 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-25T05:36:41.546 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-25T05:36:41.692 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:41.774 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:46 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:41.941 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:42.957 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:42.957 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:47 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:43.173 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-01-25T05:36:43.174 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:43.174 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2023-01-25T05:36:43.174 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:43.175 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-01-25T05:36:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 179 k 2023-01-25T05:36:43.175 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:43.175 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-01-25T05:36:43.175 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-01-25T05:36:43.176 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2023-01-25T05:36:43.176 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:43.176 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 179 k 2023-01-25T05:36:43.176 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 568 k 2023-01-25T05:36:43.176 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-01-25T05:36:43.379 INFO:teuthology.orchestra.run.smithi077.stdout:rbd-nbd-18.0.0-1959.gb0d25925.el8.x86_64.rpm 879 kB/s | 179 kB 00:00 2023-01-25T05:36:43.380 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:36:43.380 INFO:teuthology.orchestra.run.smithi077.stdout:Total 874 kB/s | 179 kB 00:00 2023-01-25T05:36:43.380 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-01-25T05:36:43.391 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-01-25T05:36:43.391 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-01-25T05:36:43.446 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-01-25T05:36:43.450 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-01-25T05:36:43.568 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-01-25T05:36:43.743 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:43.976 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:44.997 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:36:44.998 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:03:49 ago on Wed 25 Jan 2023 05:32:55 AM UTC. 2023-01-25T05:36:45.298 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-25T05:36:45.481 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:45.481 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-25T05:36:45.481 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:36:45.481 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:45.482 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-25T05:36:45.603 DEBUG:teuthology.parallel:result is None 2023-01-25T05:36:47.405 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-01-25T05:36:47.586 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:47.586 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-01-25T05:36:47.587 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:36:47.587 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:47.587 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-01-25T05:36:47.718 DEBUG:teuthology.parallel:result is None 2023-01-25T05:36:47.719 DEBUG:teuthology.orchestra.run.smithi077:> sudo lsb_release -is 2023-01-25T05:36:47.761 INFO:teuthology.orchestra.run.smithi077.stdout:RedHatEnterprise 2023-01-25T05:36:47.762 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:36:47.763 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:36:47.917 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T05:36:47.938 INFO:teuthology.orchestra.run.smithi077.stdout:18.0.0-1959.gb0d25925.el8 2023-01-25T05:36:47.939 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1959.gb0d25925.el8 2023-01-25T05:36:47.940 INFO:teuthology.task.install:The correct ceph version 18.0.0-1959.gb0d25925 is installed. 2023-01-25T05:36:47.940 DEBUG:teuthology.orchestra.run.smithi121:> sudo lsb_release -is 2023-01-25T05:36:47.990 INFO:teuthology.orchestra.run.smithi121.stdout:RedHatEnterprise 2023-01-25T05:36:47.992 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:36:47.992 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:36:48.143 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T05:36:48.168 INFO:teuthology.orchestra.run.smithi121.stdout:18.0.0-1959.gb0d25925.el8 2023-01-25T05:36:48.169 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1959.gb0d25925.el8 2023-01-25T05:36:48.169 INFO:teuthology.task.install:The correct ceph version 18.0.0-1959.gb0d25925 is installed. 2023-01-25T05:36:48.170 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-25T05:36:48.171 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:36:48.171 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T05:36:48.199 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:36:48.199 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T05:36:48.239 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-25T05:36:48.240 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:36:48.240 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T05:36:48.267 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T05:36:48.332 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:36:48.333 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T05:36:48.365 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T05:36:48.435 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-25T05:36:48.436 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:36:48.436 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T05:36:48.463 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T05:36:48.528 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:36:48.529 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T05:36:48.557 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T05:36:48.626 INFO:teuthology.run_tasks:Running task ceph... 2023-01-25T05:36:48.695 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-25T05:36:48.696 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /var/log/ceph 2023-01-25T05:36:48.699 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /var/log/ceph 2023-01-25T05:36:48.726 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-25T05:36:48.727 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T05:36:48.765 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T05:36:48.793 INFO:tasks.ceph:Creating extra log directories... 2023-01-25T05:36:48.794 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T05:36:48.831 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T05:36:48.872 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-25T05:36:48.872 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-25T05:36:48.873 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136832', 'branch': 'wip-yuri-testing-2023-01-23-0926', 'description': 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-snaps}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7136832', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-01-23-0926'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri-testing-2023-01-23-0926', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri-testing-2023-01-23-0926', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'targets': {'smithi077.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi121.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'cluster': 'ceph'}}, {'thrashosds': {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache target_max_objects 250', 'sudo ceph osd pool set cache min_read_recency_for_promote 2']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_22:20:59', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-25T05:36:48.873 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T05:36:48.897 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T05:36:48.918 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T05:36:48.954 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T05:36:48.984 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:36:48.985 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T05:36:49.035 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T05:36:49.036 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2023-01-25T05:36:49.092 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T05:36:49.092 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:36:49.093 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 351951 Links: 1 2023-01-25T05:36:49.093 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:36:49.093 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:36:49.093 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2023-01-25 05:36:36.684978382 +0000 2023-01-25T05:36:49.093 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2023-01-25 05:32:10.773431408 +0000 2023-01-25T05:36:49.094 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2023-01-25 05:32:10.773431408 +0000 2023-01-25T05:36:49.094 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2023-01-25T05:36:49.094 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T05:36:49.157 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2023-01-25T05:36:49.157 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2023-01-25T05:36:49.158 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000261249 s, 2.0 MB/s 2023-01-25T05:36:49.159 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T05:36:49.217 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2023-01-25T05:36:49.273 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T05:36:49.273 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:36:49.274 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 357748 Links: 1 2023-01-25T05:36:49.274 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:36:49.275 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:36:49.275 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2023-01-25 05:36:36.684978382 +0000 2023-01-25T05:36:49.275 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2023-01-25 05:32:11.265420320 +0000 2023-01-25T05:36:49.275 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2023-01-25 05:32:11.265420320 +0000 2023-01-25T05:36:49.275 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2023-01-25T05:36:49.276 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T05:36:49.338 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2023-01-25T05:36:49.338 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2023-01-25T05:36:49.339 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000220054 s, 2.3 MB/s 2023-01-25T05:36:49.339 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T05:36:49.397 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2023-01-25T05:36:49.454 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T05:36:49.454 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:36:49.455 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 355300 Links: 1 2023-01-25T05:36:49.455 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:36:49.455 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:36:49.455 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2023-01-25 05:36:36.684978382 +0000 2023-01-25T05:36:49.455 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2023-01-25 05:32:11.759409187 +0000 2023-01-25T05:36:49.456 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2023-01-25 05:32:11.759409187 +0000 2023-01-25T05:36:49.456 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2023-01-25T05:36:49.456 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T05:36:49.519 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2023-01-25T05:36:49.519 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2023-01-25T05:36:49.519 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000224499 s, 2.3 MB/s 2023-01-25T05:36:49.520 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T05:36:49.578 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2023-01-25T05:36:49.636 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T05:36:49.636 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:36:49.636 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 358698 Links: 1 2023-01-25T05:36:49.637 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:36:49.637 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:36:49.637 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2023-01-25 05:36:36.684978382 +0000 2023-01-25T05:36:49.637 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2023-01-25 05:32:12.262397851 +0000 2023-01-25T05:36:49.637 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2023-01-25 05:32:12.262397851 +0000 2023-01-25T05:36:49.638 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2023-01-25T05:36:49.638 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T05:36:49.701 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2023-01-25T05:36:49.701 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2023-01-25T05:36:49.701 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000170916 s, 3.0 MB/s 2023-01-25T05:36:49.702 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T05:36:49.760 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:36:49.761 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:36:49.761 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T05:36:49.782 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T05:36:49.782 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2023-01-25T05:36:49.842 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T05:36:49.842 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:36:49.843 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 354782 Links: 1 2023-01-25T05:36:49.843 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:36:49.843 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:36:49.843 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-01-25 05:36:34.658698161 +0000 2023-01-25T05:36:49.843 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-01-25 05:32:10.782550290 +0000 2023-01-25T05:36:49.844 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-01-25 05:32:10.782550290 +0000 2023-01-25T05:36:49.844 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-01-25T05:36:49.844 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T05:36:49.912 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-01-25T05:36:49.913 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-01-25T05:36:49.913 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000286459 s, 1.8 MB/s 2023-01-25T05:36:49.914 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T05:36:49.973 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2023-01-25T05:36:50.033 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T05:36:50.049 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:36:50.049 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 357565 Links: 1 2023-01-25T05:36:50.050 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:36:50.050 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:36:50.050 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-01-25 05:36:34.659698148 +0000 2023-01-25T05:36:50.050 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-01-25 05:32:11.275543817 +0000 2023-01-25T05:36:50.050 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-01-25 05:32:11.275543817 +0000 2023-01-25T05:36:50.051 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-01-25T05:36:50.051 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T05:36:50.101 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-01-25T05:36:50.102 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-01-25T05:36:50.102 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000238636 s, 2.1 MB/s 2023-01-25T05:36:50.103 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T05:36:50.165 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2023-01-25T05:36:50.223 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T05:36:50.223 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:36:50.224 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 359798 Links: 1 2023-01-25T05:36:50.224 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:36:50.224 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:36:50.224 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-01-25 05:36:34.659698148 +0000 2023-01-25T05:36:50.225 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-01-25 05:32:11.761537435 +0000 2023-01-25T05:36:50.225 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-01-25 05:32:11.761537435 +0000 2023-01-25T05:36:50.225 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-01-25T05:36:50.225 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T05:36:50.290 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-01-25T05:36:50.291 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-01-25T05:36:50.291 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000224022 s, 2.3 MB/s 2023-01-25T05:36:50.293 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T05:36:50.351 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2023-01-25T05:36:50.410 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T05:36:50.411 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:36:50.411 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 358729 Links: 1 2023-01-25T05:36:50.411 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:36:50.411 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:36:50.412 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-01-25 05:36:34.659698148 +0000 2023-01-25T05:36:50.412 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-01-25 05:32:12.264530831 +0000 2023-01-25T05:36:50.412 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-01-25 05:32:12.264530831 +0000 2023-01-25T05:36:50.412 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-01-25T05:36:50.413 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T05:36:50.476 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-01-25T05:36:50.477 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-01-25T05:36:50.477 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000172489 s, 3.0 MB/s 2023-01-25T05:36:50.478 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T05:36:50.536 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:36:50.537 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi077.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T05:36:50.537 INFO:tasks.ceph:Generating config... 2023-01-25T05:36:50.538 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-25T05:36:50.539 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-01-25T05:36:50.539 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2023-01-25T05:36:50.539 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-01-25T05:36:50.540 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-01-25T05:36:50.540 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-25T05:36:50.540 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-01-25T05:36:50.540 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-01-25T05:36:50.541 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-25T05:36:50.541 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-25T05:36:50.541 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-01-25T05:36:50.541 INFO:tasks.ceph:[osd] filestore odsync write = True 2023-01-25T05:36:50.542 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-01-25T05:36:50.542 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-01-25T05:36:50.542 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-01-25T05:36:50.542 INFO:tasks.ceph:[osd] osd backoff on degraded = True 2023-01-25T05:36:50.543 INFO:tasks.ceph:[osd] osd backoff on peering = True 2023-01-25T05:36:50.543 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-01-25T05:36:50.543 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-25T05:36:50.543 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-25T05:36:50.544 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-01-25T05:36:50.544 INFO:tasks.ceph:[osd] osd max backfills = 2 2023-01-25T05:36:50.544 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-01-25T05:36:50.545 INFO:tasks.ceph:[osd] osd max scrubs = 3 2023-01-25T05:36:50.545 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-25T05:36:50.545 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-01-25T05:36:50.545 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-25T05:36:50.546 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-25T05:36:50.546 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-01-25T05:36:50.546 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-01-25T05:36:50.547 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-25T05:36:50.547 INFO:tasks.ceph:[osd] osd snap trim sleep = 0.5 2023-01-25T05:36:50.547 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-01-25T05:36:50.547 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-01-25T05:36:50.548 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-01-25T05:36:50.548 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-01-25T05:36:50.548 INFO:tasks.ceph:[global] ms inject delay max = 1 2023-01-25T05:36:50.548 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2023-01-25T05:36:50.549 INFO:tasks.ceph:[global] ms inject delay type = osd 2023-01-25T05:36:50.549 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2023-01-25T05:36:50.549 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2023-01-25T05:36:50.550 INFO:tasks.ceph:[global] ms type = async 2023-01-25T05:36:50.550 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 2 2023-01-25T05:36:50.550 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 1 2023-01-25T05:36:50.550 INFO:tasks.ceph:[global] osd_pg_log_trim_min = 0 2023-01-25T05:36:50.551 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-01-25T05:36:50.551 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2023-01-25T05:36:50.551 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-25T05:36:50.552 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-25T05:36:50.552 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-25T05:36:50.552 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-25T05:36:50.552 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-25T05:36:50.553 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-01-25T05:36:50.553 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-01-25T05:36:50.553 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-01-25T05:36:50.553 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-01-25T05:36:50.554 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-25T05:36:50.554 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-01-25T05:36:50.554 INFO:tasks.ceph:Setting up mon.a... 2023-01-25T05:36:50.555 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-25T05:36:50.624 INFO:teuthology.orchestra.run.smithi077.stdout:creating /etc/ceph/ceph.keyring 2023-01-25T05:36:50.627 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-25T05:36:50.675 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T05:36:50.702 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.77'), ('mon.c', '[v2:172.21.15.77:3301,v1:172.21.15.77:6790]'), ('mon.b', '172.21.15.121')] 2023-01-25T05:36:50.703 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790],172.21.15.121', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-25T05:36:50.704 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:36:50.704 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T05:36:50.759 DEBUG:teuthology.orchestra.run.smithi077:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.77 --addv c '[v2:172.21.15.77:3301,v1:172.21.15.77:6790]' --add b 172.21.15.121 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:36:50.847 INFO:teuthology.orchestra.run.smithi077.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:36:50.847 INFO:teuthology.orchestra.run.smithi077.stdout:monmaptool: generated fsid 7bfb8493-ac34-42fb-8797-f07adda855bd 2023-01-25T05:36:50.847 INFO:teuthology.orchestra.run.smithi077.stdout:setting min_mon_release = pacific 2023-01-25T05:36:50.848 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 0 2023-01-25T05:36:50.848 INFO:teuthology.orchestra.run.smithi077.stdout:fsid 7bfb8493-ac34-42fb-8797-f07adda855bd 2023-01-25T05:36:50.848 INFO:teuthology.orchestra.run.smithi077.stdout:last_changed 2023-01-25T05:36:50.846912+0000 2023-01-25T05:36:50.848 INFO:teuthology.orchestra.run.smithi077.stdout:created 2023-01-25T05:36:50.846912+0000 2023-01-25T05:36:50.848 INFO:teuthology.orchestra.run.smithi077.stdout:min_mon_release 16 (pacific) 2023-01-25T05:36:50.849 INFO:teuthology.orchestra.run.smithi077.stdout:election_strategy: 3 2023-01-25T05:36:50.849 INFO:teuthology.orchestra.run.smithi077.stdout:0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2023-01-25T05:36:50.849 INFO:teuthology.orchestra.run.smithi077.stdout:1: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.b 2023-01-25T05:36:50.849 INFO:teuthology.orchestra.run.smithi077.stdout:2: [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] mon.c 2023-01-25T05:36:50.849 INFO:teuthology.orchestra.run.smithi077.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-25T05:36:50.850 DEBUG:teuthology.orchestra.run.smithi077:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T05:36:50.867 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7bfb8493-ac34-42fb-8797-f07adda855bd... 2023-01-25T05:36:50.868 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T05:36:50.910 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T05:36:50.952 INFO:teuthology.orchestra.run.smithi077.stdout:[global] 2023-01-25T05:36:50.953 INFO:teuthology.orchestra.run.smithi077.stdout: chdir = "" 2023-01-25T05:36:50.953 INFO:teuthology.orchestra.run.smithi077.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T05:36:50.953 INFO:teuthology.orchestra.run.smithi077.stdout: auth supported = cephx 2023-01-25T05:36:50.953 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.953 INFO:teuthology.orchestra.run.smithi077.stdout: filestore xattr use omap = true 2023-01-25T05:36:50.954 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.954 INFO:teuthology.orchestra.run.smithi077.stdout: mon clock drift allowed = 1.000 2023-01-25T05:36:50.954 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.954 INFO:teuthology.orchestra.run.smithi077.stdout: osd crush chooseleaf type = 0 2023-01-25T05:36:50.954 INFO:teuthology.orchestra.run.smithi077.stdout: auth debug = true 2023-01-25T05:36:50.955 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.955 INFO:teuthology.orchestra.run.smithi077.stdout: ms die on old message = true 2023-01-25T05:36:50.955 INFO:teuthology.orchestra.run.smithi077.stdout: ms die on bug = true 2023-01-25T05:36:50.955 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.955 INFO:teuthology.orchestra.run.smithi077.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T05:36:50.956 INFO:teuthology.orchestra.run.smithi077.stdout: mon pg warn max object skew = 0 2023-01-25T05:36:50.956 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.956 INFO:teuthology.orchestra.run.smithi077.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T05:36:50.956 INFO:teuthology.orchestra.run.smithi077.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T05:36:50.956 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.957 INFO:teuthology.orchestra.run.smithi077.stdout: osd pool default size = 2 2023-01-25T05:36:50.957 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.957 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd allow primary affinity = true 2023-01-25T05:36:50.957 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd allow pg remap = true 2023-01-25T05:36:50.957 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on legacy crush tunables = false 2023-01-25T05:36:50.958 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on crush straw calc version zero = false 2023-01-25T05:36:50.958 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on no sortbitwise = false 2023-01-25T05:36:50.958 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on osd down out interval zero = false 2023-01-25T05:36:50.958 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on too few osds = false 2023-01-25T05:36:50.958 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T05:36:50.959 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T05:36:50.959 INFO:teuthology.orchestra.run.smithi077.stdout: mon_allow_pool_size_one = true 2023-01-25T05:36:50.959 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.959 INFO:teuthology.orchestra.run.smithi077.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T05:36:50.959 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.959 INFO:teuthology.orchestra.run.smithi077.stdout: osd default data pool replay window = 5 2023-01-25T05:36:50.960 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.960 INFO:teuthology.orchestra.run.smithi077.stdout: mon allow pool delete = true 2023-01-25T05:36:50.960 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.960 INFO:teuthology.orchestra.run.smithi077.stdout: mon cluster log file level = debug 2023-01-25T05:36:50.961 INFO:teuthology.orchestra.run.smithi077.stdout: debug asserts on shutdown = true 2023-01-25T05:36:50.961 INFO:teuthology.orchestra.run.smithi077.stdout: mon health detail to clog = false 2023-01-25T05:36:50.961 INFO:teuthology.orchestra.run.smithi077.stdout: mon host = "172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790],172.21.15.121" 2023-01-25T05:36:50.961 INFO:teuthology.orchestra.run.smithi077.stdout: mon client directed command retry = 5 2023-01-25T05:36:50.961 INFO:teuthology.orchestra.run.smithi077.stdout: mon election default strategy = 3 2023-01-25T05:36:50.962 INFO:teuthology.orchestra.run.smithi077.stdout: ms inject delay max = 1 2023-01-25T05:36:50.962 INFO:teuthology.orchestra.run.smithi077.stdout: ms inject delay probability = 0.005 2023-01-25T05:36:50.962 INFO:teuthology.orchestra.run.smithi077.stdout: ms inject delay type = osd 2023-01-25T05:36:50.962 INFO:teuthology.orchestra.run.smithi077.stdout: ms inject internal delays = 0.002 2023-01-25T05:36:50.962 INFO:teuthology.orchestra.run.smithi077.stdout: ms inject socket failures = 2500 2023-01-25T05:36:50.963 INFO:teuthology.orchestra.run.smithi077.stdout: ms type = async 2023-01-25T05:36:50.963 INFO:teuthology.orchestra.run.smithi077.stdout: osd_max_pg_log_entries = 2 2023-01-25T05:36:50.963 INFO:teuthology.orchestra.run.smithi077.stdout: osd_min_pg_log_entries = 1 2023-01-25T05:36:50.963 INFO:teuthology.orchestra.run.smithi077.stdout: osd_pg_log_trim_min = 0 2023-01-25T05:36:50.964 INFO:teuthology.orchestra.run.smithi077.stdout: osd_pool_default_min_size = 2 2023-01-25T05:36:50.964 INFO:teuthology.orchestra.run.smithi077.stdout: osd_pool_default_size = 3 2023-01-25T05:36:50.964 INFO:teuthology.orchestra.run.smithi077.stdout: fsid = 7bfb8493-ac34-42fb-8797-f07adda855bd 2023-01-25T05:36:50.964 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.964 INFO:teuthology.orchestra.run.smithi077.stdout:[osd] 2023-01-25T05:36:50.965 INFO:teuthology.orchestra.run.smithi077.stdout: osd journal size = 100 2023-01-25T05:36:50.965 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.965 INFO:teuthology.orchestra.run.smithi077.stdout: osd scrub load threshold = 5.0 2023-01-25T05:36:50.965 INFO:teuthology.orchestra.run.smithi077.stdout: osd scrub max interval = 120 2023-01-25T05:36:50.965 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.966 INFO:teuthology.orchestra.run.smithi077.stdout: osd recover clone overlap = true 2023-01-25T05:36:50.966 INFO:teuthology.orchestra.run.smithi077.stdout: osd recovery max chunk = 1048576 2023-01-25T05:36:50.966 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.966 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug shutdown = true 2023-01-25T05:36:50.966 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug op order = true 2023-01-25T05:36:50.967 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug verify stray on activate = true 2023-01-25T05:36:50.967 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.967 INFO:teuthology.orchestra.run.smithi077.stdout: osd open classes on start = true 2023-01-25T05:36:50.967 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug pg log writeout = true 2023-01-25T05:36:50.967 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.968 INFO:teuthology.orchestra.run.smithi077.stdout: osd deep scrub update digest min age = 30 2023-01-25T05:36:50.968 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.968 INFO:teuthology.orchestra.run.smithi077.stdout: osd map max advance = 10 2023-01-25T05:36:50.968 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.968 INFO:teuthology.orchestra.run.smithi077.stdout: journal zero on create = true 2023-01-25T05:36:50.969 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.969 INFO:teuthology.orchestra.run.smithi077.stdout: filestore ondisk finisher threads = 3 2023-01-25T05:36:50.969 INFO:teuthology.orchestra.run.smithi077.stdout: filestore apply finisher threads = 3 2023-01-25T05:36:50.969 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.969 INFO:teuthology.orchestra.run.smithi077.stdout: bdev debug aio = true 2023-01-25T05:36:50.970 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug misdirected ops = true 2023-01-25T05:36:50.970 INFO:teuthology.orchestra.run.smithi077.stdout: debug monc = 20 2023-01-25T05:36:50.970 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore block size = 96636764160 2023-01-25T05:36:50.970 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore compression algorithm = snappy 2023-01-25T05:36:50.970 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore compression mode = aggressive 2023-01-25T05:36:50.971 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore fsck on mount = True 2023-01-25T05:36:50.971 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore zero block detection = True 2023-01-25T05:36:50.971 INFO:teuthology.orchestra.run.smithi077.stdout: debug bluefs = 20 2023-01-25T05:36:50.971 INFO:teuthology.orchestra.run.smithi077.stdout: debug bluestore = 20 2023-01-25T05:36:50.971 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2023-01-25T05:36:50.972 INFO:teuthology.orchestra.run.smithi077.stdout: debug osd = 20 2023-01-25T05:36:50.972 INFO:teuthology.orchestra.run.smithi077.stdout: debug rocksdb = 10 2023-01-25T05:36:50.972 INFO:teuthology.orchestra.run.smithi077.stdout: filestore odsync write = True 2023-01-25T05:36:50.972 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd backfillfull_ratio = 0.85 2023-01-25T05:36:50.972 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd full ratio = 0.9 2023-01-25T05:36:50.973 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd nearfull ratio = 0.8 2023-01-25T05:36:50.973 INFO:teuthology.orchestra.run.smithi077.stdout: osd backoff on degraded = True 2023-01-25T05:36:50.973 INFO:teuthology.orchestra.run.smithi077.stdout: osd backoff on peering = True 2023-01-25T05:36:50.973 INFO:teuthology.orchestra.run.smithi077.stdout: osd blocked scrub grace period = 3600 2023-01-25T05:36:50.974 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug verify cached snaps = True 2023-01-25T05:36:50.974 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug verify missing on start = True 2023-01-25T05:36:50.974 INFO:teuthology.orchestra.run.smithi077.stdout: osd failsafe full ratio = 0.95 2023-01-25T05:36:50.974 INFO:teuthology.orchestra.run.smithi077.stdout: osd max backfills = 2 2023-01-25T05:36:50.974 INFO:teuthology.orchestra.run.smithi077.stdout: osd max markdown count = 1000 2023-01-25T05:36:50.975 INFO:teuthology.orchestra.run.smithi077.stdout: osd max scrubs = 3 2023-01-25T05:36:50.975 INFO:teuthology.orchestra.run.smithi077.stdout: osd mclock override recovery settings = True 2023-01-25T05:36:50.975 INFO:teuthology.orchestra.run.smithi077.stdout: osd objectstore = bluestore 2023-01-25T05:36:50.975 INFO:teuthology.orchestra.run.smithi077.stdout: osd op queue = debug_random 2023-01-25T05:36:50.975 INFO:teuthology.orchestra.run.smithi077.stdout: osd op queue cut off = debug_random 2023-01-25T05:36:50.976 INFO:teuthology.orchestra.run.smithi077.stdout: osd scrub min interval = 60 2023-01-25T05:36:50.976 INFO:teuthology.orchestra.run.smithi077.stdout: osd shutdown pgref assert = True 2023-01-25T05:36:50.976 INFO:teuthology.orchestra.run.smithi077.stdout: osd snap trim sleep = 0.5 2023-01-25T05:36:50.976 INFO:teuthology.orchestra.run.smithi077.stdout: osd_max_pg_log_entries = 3000 2023-01-25T05:36:50.976 INFO:teuthology.orchestra.run.smithi077.stdout: osd_min_pg_log_entries = 3000 2023-01-25T05:36:50.977 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.977 INFO:teuthology.orchestra.run.smithi077.stdout:[mgr] 2023-01-25T05:36:50.977 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2023-01-25T05:36:50.977 INFO:teuthology.orchestra.run.smithi077.stdout: debug mgr = 20 2023-01-25T05:36:50.977 INFO:teuthology.orchestra.run.smithi077.stdout: debug mon = 20 2023-01-25T05:36:50.977 INFO:teuthology.orchestra.run.smithi077.stdout: debug auth = 20 2023-01-25T05:36:50.978 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:36:50.978 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:36:50.978 INFO:teuthology.orchestra.run.smithi077.stdout: mgr/telemetry/nag = false 2023-01-25T05:36:50.978 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.978 INFO:teuthology.orchestra.run.smithi077.stdout:[mon] 2023-01-25T05:36:50.978 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2023-01-25T05:36:50.979 INFO:teuthology.orchestra.run.smithi077.stdout: debug mon = 20 2023-01-25T05:36:50.979 INFO:teuthology.orchestra.run.smithi077.stdout: debug paxos = 20 2023-01-25T05:36:50.979 INFO:teuthology.orchestra.run.smithi077.stdout: debug auth = 20 2023-01-25T05:36:50.979 INFO:teuthology.orchestra.run.smithi077.stdout: mon data avail warn = 5 2023-01-25T05:36:50.979 INFO:teuthology.orchestra.run.smithi077.stdout: mon mgr mkfs grace = 240 2023-01-25T05:36:50.980 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:36:50.980 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd reporter subtree level = osd 2023-01-25T05:36:50.980 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd prime pg temp = true 2023-01-25T05:36:50.980 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:36:50.981 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.981 INFO:teuthology.orchestra.run.smithi077.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T05:36:50.981 INFO:teuthology.orchestra.run.smithi077.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T05:36:50.981 INFO:teuthology.orchestra.run.smithi077.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T05:36:50.982 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.982 INFO:teuthology.orchestra.run.smithi077.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T05:36:50.982 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T05:36:50.982 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T05:36:50.983 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.983 INFO:teuthology.orchestra.run.smithi077.stdout: # 1m isn't quite enough 2023-01-25T05:36:50.983 INFO:teuthology.orchestra.run.smithi077.stdout: mon_down_mkfs_grace = 2m 2023-01-25T05:36:50.983 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.983 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_filestore_osds = false 2023-01-25T05:36:50.984 INFO:teuthology.orchestra.run.smithi077.stdout: mon min osdmap epochs = 50 2023-01-25T05:36:50.984 INFO:teuthology.orchestra.run.smithi077.stdout: mon osdmap full prune interval = 2 2023-01-25T05:36:50.984 INFO:teuthology.orchestra.run.smithi077.stdout: mon osdmap full prune min = 15 2023-01-25T05:36:50.984 INFO:teuthology.orchestra.run.smithi077.stdout: mon osdmap full prune txsize = 2 2023-01-25T05:36:50.984 INFO:teuthology.orchestra.run.smithi077.stdout: mon scrub interval = 300 2023-01-25T05:36:50.985 INFO:teuthology.orchestra.run.smithi077.stdout: paxos service trim min = 10 2023-01-25T05:36:50.985 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:36:50.985 INFO:teuthology.orchestra.run.smithi077.stdout:[client] 2023-01-25T05:36:50.985 INFO:teuthology.orchestra.run.smithi077.stdout: rgw cache enabled = true 2023-01-25T05:36:50.986 INFO:teuthology.orchestra.run.smithi077.stdout: rgw enable ops log = true 2023-01-25T05:36:50.986 INFO:teuthology.orchestra.run.smithi077.stdout: rgw enable usage log = true 2023-01-25T05:36:50.986 INFO:teuthology.orchestra.run.smithi077.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T05:36:50.986 INFO:teuthology.orchestra.run.smithi077.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T05:36:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.a] 2023-01-25T05:36:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.c] 2023-01-25T05:36:50.987 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.b] 2023-01-25T05:36:50.989 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2023-01-25T05:36:50.989 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2023-01-25T05:36:50.989 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T05:36:50.989 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2023-01-25T05:36:50.989 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.990 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2023-01-25T05:36:50.990 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.990 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2023-01-25T05:36:50.990 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.990 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2023-01-25T05:36:50.991 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2023-01-25T05:36:50.991 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.991 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2023-01-25T05:36:50.991 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2023-01-25T05:36:50.991 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.992 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T05:36:50.992 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2023-01-25T05:36:50.992 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.992 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T05:36:50.993 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T05:36:50.993 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.993 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2023-01-25T05:36:50.993 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.993 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2023-01-25T05:36:50.994 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2023-01-25T05:36:50.994 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2023-01-25T05:36:50.994 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2023-01-25T05:36:50.994 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2023-01-25T05:36:50.994 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2023-01-25T05:36:50.995 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2023-01-25T05:36:50.995 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T05:36:50.995 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T05:36:50.995 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2023-01-25T05:36:50.995 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.996 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T05:36:50.996 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.996 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2023-01-25T05:36:50.996 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.996 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2023-01-25T05:36:50.997 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:50.997 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2023-01-25T05:36:50.997 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2023-01-25T05:36:50.997 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2023-01-25T05:36:50.998 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = "172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790],172.21.15.121" 2023-01-25T05:36:50.998 INFO:teuthology.orchestra.run.smithi121.stdout: mon client directed command retry = 5 2023-01-25T05:36:50.998 INFO:teuthology.orchestra.run.smithi121.stdout: mon election default strategy = 3 2023-01-25T05:36:50.998 INFO:teuthology.orchestra.run.smithi121.stdout: ms inject delay max = 1 2023-01-25T05:36:50.998 INFO:teuthology.orchestra.run.smithi121.stdout: ms inject delay probability = 0.005 2023-01-25T05:36:50.999 INFO:teuthology.orchestra.run.smithi121.stdout: ms inject delay type = osd 2023-01-25T05:36:50.999 INFO:teuthology.orchestra.run.smithi121.stdout: ms inject internal delays = 0.002 2023-01-25T05:36:50.999 INFO:teuthology.orchestra.run.smithi121.stdout: ms inject socket failures = 2500 2023-01-25T05:36:50.999 INFO:teuthology.orchestra.run.smithi121.stdout: ms type = async 2023-01-25T05:36:50.999 INFO:teuthology.orchestra.run.smithi121.stdout: osd_max_pg_log_entries = 2 2023-01-25T05:36:51.000 INFO:teuthology.orchestra.run.smithi121.stdout: osd_min_pg_log_entries = 1 2023-01-25T05:36:51.000 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pg_log_trim_min = 0 2023-01-25T05:36:51.000 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_min_size = 2 2023-01-25T05:36:51.000 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_size = 3 2023-01-25T05:36:51.000 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = 7bfb8493-ac34-42fb-8797-f07adda855bd 2023-01-25T05:36:51.001 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.001 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2023-01-25T05:36:51.001 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2023-01-25T05:36:51.001 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.001 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2023-01-25T05:36:51.002 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 120 2023-01-25T05:36:51.002 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.002 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2023-01-25T05:36:51.002 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2023-01-25T05:36:51.003 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.003 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2023-01-25T05:36:51.003 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2023-01-25T05:36:51.003 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2023-01-25T05:36:51.003 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.004 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2023-01-25T05:36:51.004 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2023-01-25T05:36:51.004 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.004 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2023-01-25T05:36:51.004 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.005 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2023-01-25T05:36:51.005 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.005 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2023-01-25T05:36:51.005 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.005 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2023-01-25T05:36:51.006 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2023-01-25T05:36:51.006 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.006 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2023-01-25T05:36:51.006 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2023-01-25T05:36:51.007 INFO:teuthology.orchestra.run.smithi121.stdout: debug monc = 20 2023-01-25T05:36:51.007 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore block size = 96636764160 2023-01-25T05:36:51.007 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore compression algorithm = snappy 2023-01-25T05:36:51.007 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore compression mode = aggressive 2023-01-25T05:36:51.007 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore fsck on mount = True 2023-01-25T05:36:51.007 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore zero block detection = True 2023-01-25T05:36:51.008 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluefs = 20 2023-01-25T05:36:51.008 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluestore = 20 2023-01-25T05:36:51.008 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-01-25T05:36:51.008 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2023-01-25T05:36:51.008 INFO:teuthology.orchestra.run.smithi121.stdout: debug rocksdb = 10 2023-01-25T05:36:51.009 INFO:teuthology.orchestra.run.smithi121.stdout: filestore odsync write = True 2023-01-25T05:36:51.009 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull_ratio = 0.85 2023-01-25T05:36:51.009 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.9 2023-01-25T05:36:51.009 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.8 2023-01-25T05:36:51.009 INFO:teuthology.orchestra.run.smithi121.stdout: osd backoff on degraded = True 2023-01-25T05:36:51.009 INFO:teuthology.orchestra.run.smithi121.stdout: osd backoff on peering = True 2023-01-25T05:36:51.010 INFO:teuthology.orchestra.run.smithi121.stdout: osd blocked scrub grace period = 3600 2023-01-25T05:36:51.010 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify cached snaps = True 2023-01-25T05:36:51.010 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify missing on start = True 2023-01-25T05:36:51.010 INFO:teuthology.orchestra.run.smithi121.stdout: osd failsafe full ratio = 0.95 2023-01-25T05:36:51.010 INFO:teuthology.orchestra.run.smithi121.stdout: osd max backfills = 2 2023-01-25T05:36:51.011 INFO:teuthology.orchestra.run.smithi121.stdout: osd max markdown count = 1000 2023-01-25T05:36:51.011 INFO:teuthology.orchestra.run.smithi121.stdout: osd max scrubs = 3 2023-01-25T05:36:51.011 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock override recovery settings = True 2023-01-25T05:36:51.011 INFO:teuthology.orchestra.run.smithi121.stdout: osd objectstore = bluestore 2023-01-25T05:36:51.011 INFO:teuthology.orchestra.run.smithi121.stdout: osd op queue = debug_random 2023-01-25T05:36:51.012 INFO:teuthology.orchestra.run.smithi121.stdout: osd op queue cut off = debug_random 2023-01-25T05:36:51.012 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub min interval = 60 2023-01-25T05:36:51.012 INFO:teuthology.orchestra.run.smithi121.stdout: osd shutdown pgref assert = True 2023-01-25T05:36:51.012 INFO:teuthology.orchestra.run.smithi121.stdout: osd snap trim sleep = 0.5 2023-01-25T05:36:51.012 INFO:teuthology.orchestra.run.smithi121.stdout: osd_max_pg_log_entries = 3000 2023-01-25T05:36:51.013 INFO:teuthology.orchestra.run.smithi121.stdout: osd_min_pg_log_entries = 3000 2023-01-25T05:36:51.013 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.013 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2023-01-25T05:36:51.013 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-01-25T05:36:51.013 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2023-01-25T05:36:51.014 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2023-01-25T05:36:51.014 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2023-01-25T05:36:51.014 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:36:51.014 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:36:51.014 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2023-01-25T05:36:51.015 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.015 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2023-01-25T05:36:51.015 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-01-25T05:36:51.015 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2023-01-25T05:36:51.015 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2023-01-25T05:36:51.016 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2023-01-25T05:36:51.016 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2023-01-25T05:36:51.016 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2023-01-25T05:36:51.016 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:36:51.016 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2023-01-25T05:36:51.017 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2023-01-25T05:36:51.017 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:36:51.017 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.017 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T05:36:51.017 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T05:36:51.018 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T05:36:51.018 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.018 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T05:36:51.018 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T05:36:51.018 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T05:36:51.019 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.019 INFO:teuthology.orchestra.run.smithi121.stdout: # 1m isn't quite enough 2023-01-25T05:36:51.019 INFO:teuthology.orchestra.run.smithi121.stdout: mon_down_mkfs_grace = 2m 2023-01-25T05:36:51.019 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.019 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_filestore_osds = false 2023-01-25T05:36:51.020 INFO:teuthology.orchestra.run.smithi121.stdout: mon min osdmap epochs = 50 2023-01-25T05:36:51.020 INFO:teuthology.orchestra.run.smithi121.stdout: mon osdmap full prune interval = 2 2023-01-25T05:36:51.020 INFO:teuthology.orchestra.run.smithi121.stdout: mon osdmap full prune min = 15 2023-01-25T05:36:51.020 INFO:teuthology.orchestra.run.smithi121.stdout: mon osdmap full prune txsize = 2 2023-01-25T05:36:51.020 INFO:teuthology.orchestra.run.smithi121.stdout: mon scrub interval = 300 2023-01-25T05:36:51.021 INFO:teuthology.orchestra.run.smithi121.stdout: paxos service trim min = 10 2023-01-25T05:36:51.021 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-25T05:36:51.021 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2023-01-25T05:36:51.021 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2023-01-25T05:36:51.021 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2023-01-25T05:36:51.022 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2023-01-25T05:36:51.022 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T05:36:51.022 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T05:36:51.022 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2023-01-25T05:36:51.022 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.c] 2023-01-25T05:36:51.022 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.b] 2023-01-25T05:36:51.023 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-25T05:36:51.023 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-25T05:36:51.084 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-25T05:36:51.085 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:36:51.085 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-25T05:36:51.102 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:36:51.102 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-25T05:36:51.158 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi077.front.sepia.ceph.com 2023-01-25T05:36:51.158 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:36:51.158 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T05:36:51.159 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T05:36:51.233 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:36:51.234 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:36:51.290 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2023-01-25T05:36:51.291 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:36:51.291 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T05:36:51.291 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T05:36:51.329 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:36:51.329 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:36:51.385 INFO:tasks.ceph:Setting up mon nodes... 2023-01-25T05:36:51.386 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-25T05:36:51.386 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T05:36:51.438 INFO:teuthology.orchestra.run.smithi077.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T05:36:51.441 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T05:36:51.502 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T05:36:51.504 INFO:tasks.ceph:Setting up mds nodes... 2023-01-25T05:36:51.505 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-25T05:36:51.505 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-25T05:36:51.548 INFO:teuthology.orchestra.run.smithi077.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-25T05:36:51.560 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-25T05:36:51.604 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-25T05:36:51.616 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-25T05:36:51.617 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi077.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T05:36:51.617 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-25T05:36:51.644 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:36:51.644 INFO:tasks.ceph:role: osd.0 2023-01-25T05:36:51.644 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi077.front.sepia.ceph.com 2023-01-25T05:36:51.645 DEBUG:teuthology.orchestra.run.smithi077:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T05:36:51.718 INFO:teuthology.orchestra.run.smithi077.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:36:51.718 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:36:51.719 INFO:teuthology.orchestra.run.smithi077.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:36:51.719 INFO:teuthology.orchestra.run.smithi077.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:36:51.719 INFO:teuthology.orchestra.run.smithi077.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:36:51.720 INFO:teuthology.orchestra.run.smithi077.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:36:51.720 INFO:teuthology.orchestra.run.smithi077.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:36:51.720 INFO:teuthology.orchestra.run.smithi077.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:36:51.721 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:36:51.721 INFO:teuthology.orchestra.run.smithi077.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:36:52.147 INFO:teuthology.orchestra.run.smithi077.stdout:Discarding blocks...Done. 2023-01-25T05:36:52.149 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi077.front.sepia.ceph.com -o noatime 2023-01-25T05:36:52.150 DEBUG:teuthology.orchestra.run.smithi077:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-25T05:36:52.319 DEBUG:teuthology.orchestra.run.smithi077:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-25T05:36:52.352 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-25T05:36:52.419 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:36:52.420 INFO:tasks.ceph:role: osd.1 2023-01-25T05:36:52.420 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi077.front.sepia.ceph.com 2023-01-25T05:36:52.421 DEBUG:teuthology.orchestra.run.smithi077:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T05:36:52.493 INFO:teuthology.orchestra.run.smithi077.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:36:52.493 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:36:52.493 INFO:teuthology.orchestra.run.smithi077.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:36:52.494 INFO:teuthology.orchestra.run.smithi077.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:36:52.494 INFO:teuthology.orchestra.run.smithi077.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:36:52.494 INFO:teuthology.orchestra.run.smithi077.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:36:52.494 INFO:teuthology.orchestra.run.smithi077.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:36:52.495 INFO:teuthology.orchestra.run.smithi077.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:36:52.495 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:36:52.495 INFO:teuthology.orchestra.run.smithi077.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:36:52.918 INFO:teuthology.orchestra.run.smithi077.stdout:Discarding blocks...Done. 2023-01-25T05:36:52.920 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi077.front.sepia.ceph.com -o noatime 2023-01-25T05:36:52.920 DEBUG:teuthology.orchestra.run.smithi077:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-25T05:36:53.033 DEBUG:teuthology.orchestra.run.smithi077:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-25T05:36:53.063 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-25T05:36:53.130 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:36:53.130 INFO:tasks.ceph:role: osd.2 2023-01-25T05:36:53.131 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi077.front.sepia.ceph.com 2023-01-25T05:36:53.131 DEBUG:teuthology.orchestra.run.smithi077:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T05:36:53.204 INFO:teuthology.orchestra.run.smithi077.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:36:53.205 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:36:53.205 INFO:teuthology.orchestra.run.smithi077.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:36:53.205 INFO:teuthology.orchestra.run.smithi077.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:36:53.205 INFO:teuthology.orchestra.run.smithi077.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:36:53.205 INFO:teuthology.orchestra.run.smithi077.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:36:53.206 INFO:teuthology.orchestra.run.smithi077.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:36:53.206 INFO:teuthology.orchestra.run.smithi077.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:36:53.206 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:36:53.206 INFO:teuthology.orchestra.run.smithi077.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:36:53.650 INFO:teuthology.orchestra.run.smithi077.stdout:Discarding blocks...Done. 2023-01-25T05:36:53.652 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi077.front.sepia.ceph.com -o noatime 2023-01-25T05:36:53.652 DEBUG:teuthology.orchestra.run.smithi077:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-25T05:36:53.830 DEBUG:teuthology.orchestra.run.smithi077:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-25T05:36:53.863 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-25T05:36:53.929 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:36:53.929 INFO:tasks.ceph:role: osd.3 2023-01-25T05:36:53.929 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi077.front.sepia.ceph.com 2023-01-25T05:36:53.930 DEBUG:teuthology.orchestra.run.smithi077:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T05:36:54.001 INFO:teuthology.orchestra.run.smithi077.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:36:54.002 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:36:54.002 INFO:teuthology.orchestra.run.smithi077.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:36:54.002 INFO:teuthology.orchestra.run.smithi077.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:36:54.002 INFO:teuthology.orchestra.run.smithi077.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:36:54.003 INFO:teuthology.orchestra.run.smithi077.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:36:54.003 INFO:teuthology.orchestra.run.smithi077.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:36:54.003 INFO:teuthology.orchestra.run.smithi077.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:36:54.003 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:36:54.003 INFO:teuthology.orchestra.run.smithi077.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:36:54.437 INFO:teuthology.orchestra.run.smithi077.stdout:Discarding blocks...Done. 2023-01-25T05:36:54.439 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi077.front.sepia.ceph.com -o noatime 2023-01-25T05:36:54.440 DEBUG:teuthology.orchestra.run.smithi077:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-25T05:36:54.626 DEBUG:teuthology.orchestra.run.smithi077:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-25T05:36:54.658 DEBUG:teuthology.orchestra.run.smithi077:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:36:54.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:54.756+0000 7f2c81e775c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-25T05:36:54.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:54.756+0000 7f2c81e775c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-25T05:36:54.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:54.756+0000 7f2c81e775c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T05:36:54.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:54.756+0000 7f2c81e775c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T05:36:54.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:54.756+0000 7f2c81e775c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T05:36:54.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:54.757+0000 7f2c81e775c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-25T05:36:55.286 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:55.284+0000 7f2c81e775c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:36:56.051 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:56.049+0000 7f2c81e775c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:36:56.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:56.815+0000 7f2c81e775c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:36:57.618 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-25T05:36:57.674 DEBUG:teuthology.orchestra.run.smithi077:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:36:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:57.723+0000 7f38392725c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-25T05:36:57.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:57.723+0000 7f38392725c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-25T05:36:57.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:57.724+0000 7f38392725c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T05:36:57.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:57.724+0000 7f38392725c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T05:36:57.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:57.724+0000 7f38392725c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T05:36:57.726 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:57.724+0000 7f38392725c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-25T05:36:58.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:58.252+0000 7f38392725c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:36:59.018 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:59.017+0000 7f38392725c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:36:59.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:36:59.786+0000 7f38392725c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:00.583 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-25T05:37:00.615 DEBUG:teuthology.orchestra.run.smithi077:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:37:00.705 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:00.704+0000 7f25b37cf5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-25T05:37:00.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:00.704+0000 7f25b37cf5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-25T05:37:00.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:00.704+0000 7f25b37cf5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T05:37:00.707 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:00.704+0000 7f25b37cf5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T05:37:00.707 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:00.704+0000 7f25b37cf5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T05:37:00.707 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:00.705+0000 7f25b37cf5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-25T05:37:01.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:01.233+0000 7f25b37cf5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:01.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:01.998+0000 7f25b37cf5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:02.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:02.768+0000 7f25b37cf5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:03.569 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-25T05:37:03.601 DEBUG:teuthology.orchestra.run.smithi077:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:37:03.691 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:03.690+0000 7f5706bde5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-25T05:37:03.692 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:03.691+0000 7f5706bde5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-25T05:37:03.692 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:03.691+0000 7f5706bde5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T05:37:03.692 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:03.691+0000 7f5706bde5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T05:37:03.692 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:03.691+0000 7f5706bde5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T05:37:03.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:03.691+0000 7f5706bde5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-25T05:37:04.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:04.220+0000 7f5706bde5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:04.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:04.984+0000 7f5706bde5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:05.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-01-25T05:37:05.754+0000 7f5706bde5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:06.554 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-25T05:37:06.586 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-25T05:37:06.620 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:37:06.620 INFO:tasks.ceph:role: osd.4 2023-01-25T05:37:06.620 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com 2023-01-25T05:37:06.621 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T05:37:06.697 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:37:06.697 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:37:06.697 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:37:06.698 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:37:06.698 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:37:06.698 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:37:06.698 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:37:06.698 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:37:06.698 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:37:06.699 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:37:07.127 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2023-01-25T05:37:07.129 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-01-25T05:37:07.129 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-25T05:37:07.311 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-25T05:37:07.349 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-25T05:37:07.418 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:37:07.418 INFO:tasks.ceph:role: osd.5 2023-01-25T05:37:07.419 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com 2023-01-25T05:37:07.419 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T05:37:07.494 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:37:07.494 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:37:07.495 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:37:07.495 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:37:07.495 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:37:07.495 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:37:07.495 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:37:07.496 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:37:07.496 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:37:07.496 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:37:07.918 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2023-01-25T05:37:07.920 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-01-25T05:37:07.920 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-25T05:37:08.030 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-25T05:37:08.067 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-25T05:37:08.134 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:37:08.134 INFO:tasks.ceph:role: osd.6 2023-01-25T05:37:08.135 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com 2023-01-25T05:37:08.135 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T05:37:08.208 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:37:08.209 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:37:08.209 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:37:08.209 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:37:08.209 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:37:08.210 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:37:08.210 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:37:08.210 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:37:08.210 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:37:08.210 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:37:08.665 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2023-01-25T05:37:08.666 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-01-25T05:37:08.667 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-25T05:37:08.837 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-25T05:37:08.870 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-25T05:37:08.937 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:37:08.938 INFO:tasks.ceph:role: osd.7 2023-01-25T05:37:08.938 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com 2023-01-25T05:37:08.938 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T05:37:09.012 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:37:09.012 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:37:09.013 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:37:09.013 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:37:09.013 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:37:09.013 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:37:09.013 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:37:09.014 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:37:09.014 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:37:09.014 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:37:09.444 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2023-01-25T05:37:09.446 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-01-25T05:37:09.446 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-25T05:37:09.603 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-25T05:37:09.635 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:37:09.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:09.733+0000 7fa5a697b5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-25T05:37:09.734 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:09.733+0000 7fa5a697b5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-25T05:37:09.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:09.733+0000 7fa5a697b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T05:37:09.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:09.733+0000 7fa5a697b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T05:37:09.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:09.733+0000 7fa5a697b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T05:37:09.735 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:09.733+0000 7fa5a697b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-25T05:37:10.261 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:10.261+0000 7fa5a697b5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:11.030 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:11.028+0000 7fa5a697b5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:11.807 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:11.807+0000 7fa5a697b5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:12.616 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-25T05:37:12.668 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:37:12.719 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:12.718+0000 7f9b3f6be5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-25T05:37:12.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:12.718+0000 7f9b3f6be5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-25T05:37:12.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:12.718+0000 7f9b3f6be5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T05:37:12.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:12.718+0000 7f9b3f6be5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T05:37:12.720 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:12.718+0000 7f9b3f6be5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T05:37:12.721 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:12.719+0000 7f9b3f6be5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-25T05:37:13.245 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:13.244+0000 7f9b3f6be5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:14.009 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:14.009+0000 7f9b3f6be5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:14.775 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:14.774+0000 7f9b3f6be5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:15.572 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-25T05:37:15.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:37:15.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:15.694+0000 7f3a61de65c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-25T05:37:15.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:15.694+0000 7f3a61de65c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-25T05:37:15.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:15.694+0000 7f3a61de65c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T05:37:15.696 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:15.694+0000 7f3a61de65c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T05:37:15.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:15.694+0000 7f3a61de65c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T05:37:15.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:15.695+0000 7f3a61de65c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-25T05:37:16.228 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:16.227+0000 7f3a61de65c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:16.993 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:16.992+0000 7f3a61de65c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:17.762 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:17.762+0000 7f3a61de65c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:18.566 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-25T05:37:18.598 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:37:18.689 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:18.688+0000 7f8fddf635c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-25T05:37:18.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:18.688+0000 7f8fddf635c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-25T05:37:18.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:18.688+0000 7f8fddf635c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T05:37:18.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:18.688+0000 7f8fddf635c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T05:37:18.690 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:18.688+0000 7f8fddf635c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T05:37:18.691 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:18.689+0000 7f8fddf635c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-25T05:37:19.217 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:19.216+0000 7f8fddf635c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:19.984 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:19.983+0000 7f8fddf635c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:20.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-25T05:37:20.753+0000 7f8fddf635c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T05:37:21.556 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-25T05:37:21.587 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-25T05:37:21.587 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:37:21.588 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-25T05:37:21.621 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:37:21.622 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-25T05:37:21.691 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:37:21.691 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-25T05:37:21.759 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:37:21.760 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-25T05:37:21.827 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:37:21.828 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-25T05:37:21.895 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:37:21.895 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-25T05:37:21.927 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:37:21.927 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-25T05:37:21.997 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:37:21.998 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-25T05:37:22.065 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:37:22.066 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-25T05:37:22.134 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:37:22.135 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-25T05:37:22.203 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:37:22.203 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-25T05:37:22.222 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:37:22.222 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-25T05:37:22.260 INFO:tasks.ceph:Adding keys to all mons... 2023-01-25T05:37:22.260 DEBUG:teuthology.orchestra.run.smithi077:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T05:37:22.264 DEBUG:teuthology.orchestra.run.smithi121:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T05:37:22.303 INFO:teuthology.orchestra.run.smithi077.stdout:[mgr.y] 2023-01-25T05:37:22.303 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQDzv9Bj5dwiGhAAE8gaNfliy+gVk+1fgbgUFg== 2023-01-25T05:37:22.303 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.0] 2023-01-25T05:37:22.304 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQD2v9BjR3AoLRAAV6ZEP4VBXPQe94nVddhLzA== 2023-01-25T05:37:22.304 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.1] 2023-01-25T05:37:22.304 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQD5v9BjRQ4sKxAAZgPXdY0eBaPMJtqeOuMZbg== 2023-01-25T05:37:22.304 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.2] 2023-01-25T05:37:22.305 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQD8v9Bj2HYFKhAATmDJjWhZTVer8claHivrcw== 2023-01-25T05:37:22.305 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.3] 2023-01-25T05:37:22.305 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQD/v9Bj7pU1KRAAECAzFu2krqLUb6BgLOd0Nw== 2023-01-25T05:37:22.305 INFO:teuthology.orchestra.run.smithi077.stdout:[mgr.x] 2023-01-25T05:37:22.305 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQDzv9BjDgrnHRAAWaMiBG8HEoMkdl9u4i/DyQ== 2023-01-25T05:37:22.306 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.4] 2023-01-25T05:37:22.306 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQAFwNBjSLi5KxAAkNW0ORQmqyG9bX/a2iZgyw== 2023-01-25T05:37:22.306 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.5] 2023-01-25T05:37:22.306 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQAIwNBjLXncKhAAKSvZs8GgvF+Wpi6+74p2Cg== 2023-01-25T05:37:22.306 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.6] 2023-01-25T05:37:22.307 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQALwNBjbAFvKRAASaQSm1ATO04NEkyQCz7fGw== 2023-01-25T05:37:22.307 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.7] 2023-01-25T05:37:22.307 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQAOwNBjRS4SKRAAXHHp25mfeHN97awe5iaxYQ== 2023-01-25T05:37:22.307 INFO:teuthology.orchestra.run.smithi077.stdout:[client.0] 2023-01-25T05:37:22.307 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQDzv9BjygOvIBAABDyI5br33Z5qfD8FdpVv8A== 2023-01-25T05:37:22.308 INFO:teuthology.orchestra.run.smithi077.stdout:[client.1] 2023-01-25T05:37:22.308 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQDzv9Bjxrb7IxAA9qT7xHelXU+IctJisnudLg== 2023-01-25T05:37:22.325 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.y] 2023-01-25T05:37:22.326 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDzv9Bj5dwiGhAAE8gaNfliy+gVk+1fgbgUFg== 2023-01-25T05:37:22.326 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.0] 2023-01-25T05:37:22.326 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQD2v9BjR3AoLRAAV6ZEP4VBXPQe94nVddhLzA== 2023-01-25T05:37:22.327 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.1] 2023-01-25T05:37:22.327 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQD5v9BjRQ4sKxAAZgPXdY0eBaPMJtqeOuMZbg== 2023-01-25T05:37:22.327 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.2] 2023-01-25T05:37:22.328 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQD8v9Bj2HYFKhAATmDJjWhZTVer8claHivrcw== 2023-01-25T05:37:22.328 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.3] 2023-01-25T05:37:22.329 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQD/v9Bj7pU1KRAAECAzFu2krqLUb6BgLOd0Nw== 2023-01-25T05:37:22.329 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.x] 2023-01-25T05:37:22.329 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDzv9BjDgrnHRAAWaMiBG8HEoMkdl9u4i/DyQ== 2023-01-25T05:37:22.330 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.4] 2023-01-25T05:37:22.330 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAFwNBjSLi5KxAAkNW0ORQmqyG9bX/a2iZgyw== 2023-01-25T05:37:22.330 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.5] 2023-01-25T05:37:22.331 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAIwNBjLXncKhAAKSvZs8GgvF+Wpi6+74p2Cg== 2023-01-25T05:37:22.331 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.6] 2023-01-25T05:37:22.331 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQALwNBjbAFvKRAASaQSm1ATO04NEkyQCz7fGw== 2023-01-25T05:37:22.332 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.7] 2023-01-25T05:37:22.332 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAOwNBjRS4SKRAAXHHp25mfeHN97awe5iaxYQ== 2023-01-25T05:37:22.333 INFO:teuthology.orchestra.run.smithi121.stdout:[client.0] 2023-01-25T05:37:22.333 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDzv9BjygOvIBAABDyI5br33Z5qfD8FdpVv8A== 2023-01-25T05:37:22.333 INFO:teuthology.orchestra.run.smithi121.stdout:[client.1] 2023-01-25T05:37:22.334 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDzv9Bjxrb7IxAA9qT7xHelXU+IctJisnudLg== 2023-01-25T05:37:22.336 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:37:22.355 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:37:22.431 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.434 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.489 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.492 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.546 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.549 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.601 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.657 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:37:22.661 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:37:22.715 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.718 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.770 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.773 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.826 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.828 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.881 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.883 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:37:22.937 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:37:22.939 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:37:22.994 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:37:22.997 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:37:23.052 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-25T05:37:23.053 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-25T05:37:23.079 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T05:37:23.176 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-25T05:37:23.203 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-25T05:37:23.270 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T05:37:23.368 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-25T05:37:23.394 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-25T05:37:23.423 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T05:37:23.523 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-25T05:37:23.551 DEBUG:teuthology.orchestra.run.smithi077:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:37:23.553 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:37:23.607 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-25T05:37:23.607 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-25T05:37:23.608 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-25T05:37:23.610 INFO:tasks.ceph.mon.a:Started 2023-01-25T05:37:23.610 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-25T05:37:23.611 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-25T05:37:23.612 INFO:tasks.ceph.mon.c:Started 2023-01-25T05:37:23.613 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-25T05:37:23.613 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-25T05:37:23.649 INFO:tasks.ceph.mon.b:Started 2023-01-25T05:37:23.649 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-25T05:37:23.649 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-25T05:37:23.652 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-25T05:37:23.654 INFO:tasks.ceph.mgr.y:Started 2023-01-25T05:37:23.654 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-25T05:37:23.654 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-25T05:37:23.656 INFO:tasks.ceph.mgr.x:Started 2023-01-25T05:37:23.657 INFO:tasks.ceph:Setting crush tunables to default 2023-01-25T05:37:23.657 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --cluster ceph osd crush tunables default 2023-01-25T05:37:30.078 INFO:teuthology.orchestra.run.smithi077.stderr:adjusted tunables profile to default 2023-01-25T05:37:30.089 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-25T05:37:30.090 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:37:30.090 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-25T05:37:30.116 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:37:30.116 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-25T05:37:30.117 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:30.115+0000 7fa8a8baa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T05:37:30.142 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:37:30.142 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-25T05:37:30.184 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:30.182+0000 7fa8a8baa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T05:37:30.208 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-01-25T05:37:30.209 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-25T05:37:30.262 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:30.260+0000 7fa8a8baa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T05:37:30.277 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:37:30.278 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-25T05:37:30.310 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:37:30.310 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-25T05:37:30.343 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:30.341+0000 7fa8a8baa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T05:37:30.380 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:37:30.380 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-25T05:37:30.448 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-25T05:37:30.448 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-25T05:37:30.483 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:30.481+0000 7fa8a8baa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T05:37:30.516 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new e64ab012-13b0-40df-9047-e1cd4b1d9b30 0 2023-01-25T05:37:30.661 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:30.660+0000 7f4f25413200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T05:37:30.729 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:30.727+0000 7f4f25413200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T05:37:30.782 INFO:teuthology.orchestra.run.smithi121.stdout:0 2023-01-25T05:37:30.795 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 7e236477-7b67-43a9-8810-3c184b1e8613 1 2023-01-25T05:37:30.806 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:30.804+0000 7f4f25413200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T05:37:30.810 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:30.808+0000 7fa8a8baa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T05:37:30.882 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:30.880+0000 7fa8a8baa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T05:37:30.888 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:30.887+0000 7f4f25413200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T05:37:31.005 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:31.003+0000 7fa8a8baa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T05:37:31.019 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-01-25T05:37:31.032 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 4c7d5d87-fced-4c70-9ead-7b29cc05284d 2 2023-01-25T05:37:31.034 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:31.032+0000 7f4f25413200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T05:37:31.084 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:31.082+0000 7fa8a8baa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T05:37:31.303 INFO:teuthology.orchestra.run.smithi121.stdout:2 2023-01-25T05:37:31.315 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 4aff67d4-8f5d-4549-af8a-c815755ce937 3 2023-01-25T05:37:31.370 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:31.368+0000 7f4f25413200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T05:37:31.446 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:31.444+0000 7f4f25413200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T05:37:31.503 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:31.501+0000 7fa8a8baa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T05:37:31.535 INFO:teuthology.orchestra.run.smithi121.stdout:3 2023-01-25T05:37:31.548 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new b71752e2-4d8a-4c45-b193-ad8aa1fc6d87 4 2023-01-25T05:37:31.571 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:31.569+0000 7f4f25413200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T05:37:31.654 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:31.653+0000 7f4f25413200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T05:37:31.767 INFO:teuthology.orchestra.run.smithi121.stdout:4 2023-01-25T05:37:31.779 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 6ad77028-3080-4e08-871b-d47b5744fe60 5 2023-01-25T05:37:32.037 INFO:teuthology.orchestra.run.smithi121.stdout:5 2023-01-25T05:37:32.050 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new d2768cf4-a254-4169-919b-7fb789968ce7 6 2023-01-25T05:37:32.077 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:32.075+0000 7fa8a8baa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T05:37:32.123 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:32.122+0000 7f4f25413200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T05:37:32.229 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:32.227+0000 7fa8a8baa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T05:37:32.271 INFO:teuthology.orchestra.run.smithi121.stdout:6 2023-01-25T05:37:32.283 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new d5293edc-39b4-4fbe-88b6-1b1f4ca95390 7 2023-01-25T05:37:32.518 INFO:teuthology.orchestra.run.smithi121.stdout:7 2023-01-25T05:37:32.539 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-25T05:37:32.540 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-25T05:37:32.542 INFO:tasks.ceph.osd.0:Started 2023-01-25T05:37:32.542 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-25T05:37:32.543 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-25T05:37:32.545 INFO:tasks.ceph.osd.1:Started 2023-01-25T05:37:32.545 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-25T05:37:32.545 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-25T05:37:32.547 INFO:tasks.ceph.osd.2:Started 2023-01-25T05:37:32.548 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-25T05:37:32.548 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-25T05:37:32.551 INFO:tasks.ceph.osd.3:Started 2023-01-25T05:37:32.551 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-25T05:37:32.552 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-25T05:37:32.556 INFO:tasks.ceph.osd.4:Started 2023-01-25T05:37:32.556 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-25T05:37:32.557 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-25T05:37:32.559 INFO:tasks.ceph.osd.5:Started 2023-01-25T05:37:32.559 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-25T05:37:32.560 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-25T05:37:32.562 INFO:tasks.ceph.osd.6:Started 2023-01-25T05:37:32.562 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-25T05:37:32.563 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-25T05:37:32.564 INFO:tasks.ceph.osd.7:Started 2023-01-25T05:37:32.565 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T05:37:32.741 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:32.740+0000 7f4f25413200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T05:37:32.797 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:37:32.797 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":10,"fsid":"7bfb8493-ac34-42fb-8797-f07adda855bd","created":"2023-01-25T05:37:29.945290+0000","modified":"2023-01-25T05:37:32.512262+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T05:37:32.512262+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"e64ab012-13b0-40df-9047-e1cd4b1d9b30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7e236477-7b67-43a9-8810-3c184b1e8613","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4c7d5d87-fced-4c70-9ead-7b29cc05284d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4aff67d4-8f5d-4549-af8a-c815755ce937","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b71752e2-4d8a-4c45-b193-ad8aa1fc6d87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6ad77028-3080-4e08-871b-d47b5744fe60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d2768cf4-a254-4169-919b-7fb789968ce7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d5293edc-39b4-4fbe-88b6-1b1f4ca95390","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:37:32.810 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-25T05:37:32.811 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-25T05:37:32.894 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:32.892+0000 7f4f25413200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T05:37:32.956 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:32.954+0000 7fa8a8baa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:37:33.031 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:33.030+0000 7fa8a8baa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T05:37:33.155 INFO:tasks.ceph.osd.1.smithi077.stderr:2023-01-25T05:37:33.153+0000 7fbd819485c0 -1 Falling back to public interface 2023-01-25T05:37:33.163 INFO:tasks.ceph.osd.2.smithi077.stderr:2023-01-25T05:37:33.161+0000 7f398f3b15c0 -1 Falling back to public interface 2023-01-25T05:37:33.168 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:33.166+0000 7fa8a8baa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:37:33.171 INFO:tasks.ceph.osd.0.smithi077.stderr:2023-01-25T05:37:33.169+0000 7f3fc102a5c0 -1 Falling back to public interface 2023-01-25T05:37:33.171 INFO:tasks.ceph.osd.3.smithi077.stderr:2023-01-25T05:37:33.169+0000 7f0338b405c0 -1 Falling back to public interface 2023-01-25T05:37:33.172 INFO:tasks.ceph.osd.3.smithi077.stderr:2023-01-25T05:37:33.169+0000 7f032fefe700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2023-01-25T05:37:33.177 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-25T05:37:33.175+0000 7f498eafd5c0 -1 Falling back to public interface 2023-01-25T05:37:33.189 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-01-25T05:37:33.187+0000 7f093a1de5c0 -1 Falling back to public interface 2023-01-25T05:37:33.190 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-01-25T05:37:33.187+0000 7fd5dcf845c0 -1 Falling back to public interface 2023-01-25T05:37:33.191 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-01-25T05:37:33.189+0000 7fd5d4342700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6821/0: (98) Address already in use 2023-01-25T05:37:33.209 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-25T05:37:33.207+0000 7f7e775ad5c0 -1 Falling back to public interface 2023-01-25T05:37:33.325 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:33.323+0000 7fa8a8baa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T05:37:33.456 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:33.454+0000 7fa8a8baa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T05:37:33.573 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:33.571+0000 7f4f25413200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:37:33.617 DEBUG:teuthology.orchestra.run.smithi077:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T05:37:33.648 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:33.646+0000 7f4f25413200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T05:37:33.785 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:33.784+0000 7f4f25413200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:37:33.809 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:33.808+0000 7fa8a8baa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T05:37:33.811 INFO:teuthology.misc.health.smithi077.stdout: 2023-01-25T05:37:33.812 INFO:teuthology.misc.health.smithi077.stdout:{"epoch":10,"fsid":"7bfb8493-ac34-42fb-8797-f07adda855bd","created":"2023-01-25T05:37:29.945290+0000","modified":"2023-01-25T05:37:32.512262+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T05:37:32.512262+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"e64ab012-13b0-40df-9047-e1cd4b1d9b30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7e236477-7b67-43a9-8810-3c184b1e8613","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4c7d5d87-fced-4c70-9ead-7b29cc05284d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4aff67d4-8f5d-4549-af8a-c815755ce937","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b71752e2-4d8a-4c45-b193-ad8aa1fc6d87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6ad77028-3080-4e08-871b-d47b5744fe60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d2768cf4-a254-4169-919b-7fb789968ce7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d5293edc-39b4-4fbe-88b6-1b1f4ca95390","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:37:33.823 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-25T05:37:33.943 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:33.941+0000 7f4f25413200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T05:37:33.961 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:33.959+0000 7fa8a8baa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T05:37:34.075 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:34.073+0000 7fa8a8baa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T05:37:34.076 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:34.073+0000 7f4f25413200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T05:37:34.218 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:34.216+0000 7fa8a8baa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T05:37:34.318 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:34.316+0000 7fa8a8baa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T05:37:34.443 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:34.440+0000 7f4f25413200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T05:37:34.594 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:34.593+0000 7f4f25413200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T05:37:34.667 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:34.666+0000 7fa8a8baa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T05:37:34.709 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:34.707+0000 7f4f25413200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T05:37:34.856 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:34.854+0000 7f4f25413200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T05:37:34.957 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:34.955+0000 7f4f25413200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T05:37:35.242 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:35.240+0000 7fa8a8baa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T05:37:35.312 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:35.310+0000 7f4f25413200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T05:37:35.901 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-25T05:37:35.900+0000 7f4f25413200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T05:37:36.268 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:36.266+0000 7fa860f2b700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-25T05:37:36.274 INFO:tasks.ceph.mgr.y.smithi077.stderr:2023-01-25T05:37:36.272+0000 7fa860f2b700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-25T05:37:36.650 INFO:tasks.ceph.osd.1.smithi077.stderr:2023-01-25T05:37:36.648+0000 7fbd819485c0 -1 osd.1 0 log_to_monitors true 2023-01-25T05:37:36.700 INFO:tasks.ceph.osd.3.smithi077.stderr:2023-01-25T05:37:36.698+0000 7f0338b405c0 -1 osd.3 0 log_to_monitors true 2023-01-25T05:37:36.716 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-25T05:37:36.714+0000 7f498eafd5c0 -1 osd.4 0 log_to_monitors true 2023-01-25T05:37:36.721 INFO:tasks.ceph.osd.0.smithi077.stderr:2023-01-25T05:37:36.719+0000 7f3fc102a5c0 -1 osd.0 0 log_to_monitors true 2023-01-25T05:37:36.726 INFO:tasks.ceph.osd.2.smithi077.stderr:2023-01-25T05:37:36.725+0000 7f398f3b15c0 -1 osd.2 0 log_to_monitors true 2023-01-25T05:37:36.759 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-01-25T05:37:36.757+0000 7fd5dcf845c0 -1 osd.7 0 log_to_monitors true 2023-01-25T05:37:36.763 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-01-25T05:37:36.761+0000 7f093a1de5c0 -1 osd.5 0 log_to_monitors true 2023-01-25T05:37:36.765 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-25T05:37:36.763+0000 7f7e775ad5c0 -1 osd.6 0 log_to_monitors true 2023-01-25T05:37:38.278 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-25T05:37:38.277+0000 7f4982eb5700 -1 osd.4 0 waiting for initial osdmap 2023-01-25T05:37:38.280 INFO:tasks.ceph.osd.0.smithi077.stderr:2023-01-25T05:37:38.278+0000 7f3fb53e2700 -1 osd.0 0 waiting for initial osdmap 2023-01-25T05:37:38.281 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-01-25T05:37:38.280+0000 7f092e596700 -1 osd.5 0 waiting for initial osdmap 2023-01-25T05:37:38.284 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-25T05:37:38.281+0000 7f497e296700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:37:38.285 INFO:tasks.ceph.osd.0.smithi077.stderr:2023-01-25T05:37:38.282+0000 7f3faffc2700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:37:38.286 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-01-25T05:37:38.283+0000 7f0929977700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:37:38.286 INFO:tasks.ceph.osd.1.smithi077.stderr:2023-01-25T05:37:38.284+0000 7fbd75d00700 -1 osd.1 0 waiting for initial osdmap 2023-01-25T05:37:38.286 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-25T05:37:38.284+0000 7f7e6b965700 -1 osd.6 0 waiting for initial osdmap 2023-01-25T05:37:38.287 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-01-25T05:37:38.284+0000 7fd5d133c700 -1 osd.7 0 waiting for initial osdmap 2023-01-25T05:37:38.287 INFO:tasks.ceph.osd.2.smithi077.stderr:2023-01-25T05:37:38.286+0000 7f3983769700 -1 osd.2 0 waiting for initial osdmap 2023-01-25T05:37:38.289 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-01-25T05:37:38.287+0000 7fd5cc71d700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:37:38.289 INFO:tasks.ceph.osd.3.smithi077.stderr:2023-01-25T05:37:38.288+0000 7f032cef8700 -1 osd.3 0 waiting for initial osdmap 2023-01-25T05:37:38.290 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-25T05:37:38.288+0000 7f7e66d46700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:37:38.290 INFO:tasks.ceph.osd.1.smithi077.stderr:2023-01-25T05:37:38.288+0000 7fbd708e0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:37:38.294 INFO:tasks.ceph.osd.2.smithi077.stderr:2023-01-25T05:37:38.292+0000 7f397e349700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:37:38.294 INFO:tasks.ceph.osd.3.smithi077.stderr:2023-01-25T05:37:38.292+0000 7f0327ad8700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:37:40.629 DEBUG:teuthology.orchestra.run.smithi077:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T05:37:40.949 INFO:teuthology.misc.health.smithi077.stdout: 2023-01-25T05:37:40.950 INFO:teuthology.misc.health.smithi077.stdout:{"epoch":14,"fsid":"7bfb8493-ac34-42fb-8797-f07adda855bd","created":"2023-01-25T05:37:29.945290+0000","modified":"2023-01-25T05:37:40.275565+0000","last_up_change":"2023-01-25T05:37:39.272391+0000","last_in_change":"2023-01-25T05:37:32.512262+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T05:37:39.311339+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"e64ab012-13b0-40df-9047-e1cd4b1d9b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":92970},{"type":"v1","addr":"172.21.15.77:6818","nonce":92970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":92970},{"type":"v1","addr":"172.21.15.77:6822","nonce":92970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":92970},{"type":"v1","addr":"172.21.15.77:6830","nonce":92970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":92970},{"type":"v1","addr":"172.21.15.77:6826","nonce":92970}]},"public_addr":"172.21.15.77:6818/92970","cluster_addr":"172.21.15.77:6822/92970","heartbeat_back_addr":"172.21.15.77:6830/92970","heartbeat_front_addr":"172.21.15.77:6826/92970","state":["exists","up"]},{"osd":1,"uuid":"7e236477-7b67-43a9-8810-3c184b1e8613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":92968},{"type":"v1","addr":"172.21.15.77:6801","nonce":92968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":92968},{"type":"v1","addr":"172.21.15.77:6803","nonce":92968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":92968},{"type":"v1","addr":"172.21.15.77:6807","nonce":92968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":92968},{"type":"v1","addr":"172.21.15.77:6805","nonce":92968}]},"public_addr":"172.21.15.77:6801/92968","cluster_addr":"172.21.15.77:6803/92968","heartbeat_back_addr":"172.21.15.77:6807/92968","heartbeat_front_addr":"172.21.15.77:6805/92968","state":["exists","up"]},{"osd":2,"uuid":"4c7d5d87-fced-4c70-9ead-7b29cc05284d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":92969},{"type":"v1","addr":"172.21.15.77:6809","nonce":92969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":92969},{"type":"v1","addr":"172.21.15.77:6811","nonce":92969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":92969},{"type":"v1","addr":"172.21.15.77:6815","nonce":92969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":92969},{"type":"v1","addr":"172.21.15.77:6813","nonce":92969}]},"public_addr":"172.21.15.77:6809/92969","cluster_addr":"172.21.15.77:6811/92969","heartbeat_back_addr":"172.21.15.77:6815/92969","heartbeat_front_addr":"172.21.15.77:6813/92969","state":["exists","up"]},{"osd":3,"uuid":"4aff67d4-8f5d-4549-af8a-c815755ce937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6817","nonce":92971},{"type":"v1","addr":"172.21.15.77:6819","nonce":92971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6821","nonce":92971},{"type":"v1","addr":"172.21.15.77:6823","nonce":92971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6829","nonce":92971},{"type":"v1","addr":"172.21.15.77:6831","nonce":92971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6825","nonce":92971},{"type":"v1","addr":"172.21.15.77:6827","nonce":92971}]},"public_addr":"172.21.15.77:6819/92971","cluster_addr":"172.21.15.77:6823/92971","heartbeat_back_addr":"172.21.15.77:6831/92971","heartbeat_front_addr":"172.21.15.77:6827/92971","state":["exists","up"]},{"osd":4,"uuid":"b71752e2-4d8a-4c45-b193-ad8aa1fc6d87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":92931},{"type":"v1","addr":"172.21.15.121:6801","nonce":92931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":92931},{"type":"v1","addr":"172.21.15.121:6803","nonce":92931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":92931},{"type":"v1","addr":"172.21.15.121:6807","nonce":92931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":92931},{"type":"v1","addr":"172.21.15.121:6805","nonce":92931}]},"public_addr":"172.21.15.121:6801/92931","cluster_addr":"172.21.15.121:6803/92931","heartbeat_back_addr":"172.21.15.121:6807/92931","heartbeat_front_addr":"172.21.15.121:6805/92931","state":["exists","up"]},{"osd":5,"uuid":"6ad77028-3080-4e08-871b-d47b5744fe60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":92932},{"type":"v1","addr":"172.21.15.121:6810","nonce":92932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":92932},{"type":"v1","addr":"172.21.15.121:6814","nonce":92932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":92932},{"type":"v1","addr":"172.21.15.121:6821","nonce":92932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":92932},{"type":"v1","addr":"172.21.15.121:6818","nonce":92932}]},"public_addr":"172.21.15.121:6810/92932","cluster_addr":"172.21.15.121:6814/92932","heartbeat_back_addr":"172.21.15.121:6821/92932","heartbeat_front_addr":"172.21.15.121:6818/92932","state":["exists","up"]},{"osd":6,"uuid":"d2768cf4-a254-4169-919b-7fb789968ce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":92962},{"type":"v1","addr":"172.21.15.121:6825","nonce":92962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":92962},{"type":"v1","addr":"172.21.15.121:6827","nonce":92962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":92962},{"type":"v1","addr":"172.21.15.121:6831","nonce":92962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":92962},{"type":"v1","addr":"172.21.15.121:6829","nonce":92962}]},"public_addr":"172.21.15.121:6825/92962","cluster_addr":"172.21.15.121:6827/92962","heartbeat_back_addr":"172.21.15.121:6831/92962","heartbeat_front_addr":"172.21.15.121:6829/92962","state":["exists","up"]},{"osd":7,"uuid":"d5293edc-39b4-4fbe-88b6-1b1f4ca95390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6809","nonce":92933},{"type":"v1","addr":"172.21.15.121:6811","nonce":92933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6813","nonce":92933},{"type":"v1","addr":"172.21.15.121:6815","nonce":92933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":92933},{"type":"v1","addr":"172.21.15.121:6823","nonce":92933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6817","nonce":92933},{"type":"v1","addr":"172.21.15.121:6819","nonce":92933}]},"public_addr":"172.21.15.121:6811/92933","cluster_addr":"172.21.15.121:6815/92933","heartbeat_back_addr":"172.21.15.121:6823/92933","heartbeat_front_addr":"172.21.15.121:6819/92933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.744869+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.634040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.712891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.678441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.707760+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.776908+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.783829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.749070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:37:40.962 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-25T05:37:40.963 INFO:tasks.ceph:Creating RBD pool 2023-01-25T05:37:40.963 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-25T05:37:41.302 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'rbd' created 2023-01-25T05:37:41.315 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-25T05:37:42.307 INFO:teuthology.orchestra.run.smithi077.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-25T05:37:42.321 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-25T05:37:42.321 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-25T05:37:42.322 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-25T05:37:42.322 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-25T05:37:42.323 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-25T05:37:42.710 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:37:42.722 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":4,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":92696},{"type":"v1","addr":"172.21.15.77:6833","nonce":92696}]},"active_addr":"172.21.15.77:6833/92696","active_change":"2023-01-25T05:37:35.256076+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":695066212}]},{"addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":3641091692}]},{"addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":3903564150}]}]}} 2023-01-25T05:37:42.726 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-25T05:37:42.727 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-25T05:37:42.727 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T05:37:43.045 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:37:43.048 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":16,"fsid":"7bfb8493-ac34-42fb-8797-f07adda855bd","created":"2023-01-25T05:37:29.945290+0000","modified":"2023-01-25T05:37:42.296126+0000","last_up_change":"2023-01-25T05:37:39.272391+0000","last_in_change":"2023-01-25T05:37:32.512262+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T05:37:39.311339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T05:37:41.294509+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e64ab012-13b0-40df-9047-e1cd4b1d9b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":92970},{"type":"v1","addr":"172.21.15.77:6818","nonce":92970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":92970},{"type":"v1","addr":"172.21.15.77:6822","nonce":92970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":92970},{"type":"v1","addr":"172.21.15.77:6830","nonce":92970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":92970},{"type":"v1","addr":"172.21.15.77:6826","nonce":92970}]},"public_addr":"172.21.15.77:6818/92970","cluster_addr":"172.21.15.77:6822/92970","heartbeat_back_addr":"172.21.15.77:6830/92970","heartbeat_front_addr":"172.21.15.77:6826/92970","state":["exists","up"]},{"osd":1,"uuid":"7e236477-7b67-43a9-8810-3c184b1e8613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":92968},{"type":"v1","addr":"172.21.15.77:6801","nonce":92968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":92968},{"type":"v1","addr":"172.21.15.77:6803","nonce":92968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":92968},{"type":"v1","addr":"172.21.15.77:6807","nonce":92968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":92968},{"type":"v1","addr":"172.21.15.77:6805","nonce":92968}]},"public_addr":"172.21.15.77:6801/92968","cluster_addr":"172.21.15.77:6803/92968","heartbeat_back_addr":"172.21.15.77:6807/92968","heartbeat_front_addr":"172.21.15.77:6805/92968","state":["exists","up"]},{"osd":2,"uuid":"4c7d5d87-fced-4c70-9ead-7b29cc05284d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":92969},{"type":"v1","addr":"172.21.15.77:6809","nonce":92969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":92969},{"type":"v1","addr":"172.21.15.77:6811","nonce":92969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":92969},{"type":"v1","addr":"172.21.15.77:6815","nonce":92969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":92969},{"type":"v1","addr":"172.21.15.77:6813","nonce":92969}]},"public_addr":"172.21.15.77:6809/92969","cluster_addr":"172.21.15.77:6811/92969","heartbeat_back_addr":"172.21.15.77:6815/92969","heartbeat_front_addr":"172.21.15.77:6813/92969","state":["exists","up"]},{"osd":3,"uuid":"4aff67d4-8f5d-4549-af8a-c815755ce937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6817","nonce":92971},{"type":"v1","addr":"172.21.15.77:6819","nonce":92971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6821","nonce":92971},{"type":"v1","addr":"172.21.15.77:6823","nonce":92971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6829","nonce":92971},{"type":"v1","addr":"172.21.15.77:6831","nonce":92971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6825","nonce":92971},{"type":"v1","addr":"172.21.15.77:6827","nonce":92971}]},"public_addr":"172.21.15.77:6819/92971","cluster_addr":"172.21.15.77:6823/92971","heartbeat_back_addr":"172.21.15.77:6831/92971","heartbeat_front_addr":"172.21.15.77:6827/92971","state":["exists","up"]},{"osd":4,"uuid":"b71752e2-4d8a-4c45-b193-ad8aa1fc6d87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":92931},{"type":"v1","addr":"172.21.15.121:6801","nonce":92931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":92931},{"type":"v1","addr":"172.21.15.121:6803","nonce":92931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":92931},{"type":"v1","addr":"172.21.15.121:6807","nonce":92931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":92931},{"type":"v1","addr":"172.21.15.121:6805","nonce":92931}]},"public_addr":"172.21.15.121:6801/92931","cluster_addr":"172.21.15.121:6803/92931","heartbeat_back_addr":"172.21.15.121:6807/92931","heartbeat_front_addr":"172.21.15.121:6805/92931","state":["exists","up"]},{"osd":5,"uuid":"6ad77028-3080-4e08-871b-d47b5744fe60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":92932},{"type":"v1","addr":"172.21.15.121:6810","nonce":92932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":92932},{"type":"v1","addr":"172.21.15.121:6814","nonce":92932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":92932},{"type":"v1","addr":"172.21.15.121:6821","nonce":92932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":92932},{"type":"v1","addr":"172.21.15.121:6818","nonce":92932}]},"public_addr":"172.21.15.121:6810/92932","cluster_addr":"172.21.15.121:6814/92932","heartbeat_back_addr":"172.21.15.121:6821/92932","heartbeat_front_addr":"172.21.15.121:6818/92932","state":["exists","up"]},{"osd":6,"uuid":"d2768cf4-a254-4169-919b-7fb789968ce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":92962},{"type":"v1","addr":"172.21.15.121:6825","nonce":92962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":92962},{"type":"v1","addr":"172.21.15.121:6827","nonce":92962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":92962},{"type":"v1","addr":"172.21.15.121:6831","nonce":92962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":92962},{"type":"v1","addr":"172.21.15.121:6829","nonce":92962}]},"public_addr":"172.21.15.121:6825/92962","cluster_addr":"172.21.15.121:6827/92962","heartbeat_back_addr":"172.21.15.121:6831/92962","heartbeat_front_addr":"172.21.15.121:6829/92962","state":["exists","up"]},{"osd":7,"uuid":"d5293edc-39b4-4fbe-88b6-1b1f4ca95390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6809","nonce":92933},{"type":"v1","addr":"172.21.15.121:6811","nonce":92933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6813","nonce":92933},{"type":"v1","addr":"172.21.15.121:6815","nonce":92933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":92933},{"type":"v1","addr":"172.21.15.121:6823","nonce":92933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6817","nonce":92933},{"type":"v1","addr":"172.21.15.121:6819","nonce":92933}]},"public_addr":"172.21.15.121:6811/92933","cluster_addr":"172.21.15.121:6815/92933","heartbeat_back_addr":"172.21.15.121:6823/92933","heartbeat_front_addr":"172.21.15.121:6819/92933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.744869+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.634040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.712891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.678441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.707760+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.776908+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.783829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.749070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:37:43.060 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-25T05:37:43.060 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T05:37:43.407 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:37:43.407 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":17,"fsid":"7bfb8493-ac34-42fb-8797-f07adda855bd","created":"2023-01-25T05:37:29.945290+0000","modified":"2023-01-25T05:37:43.305128+0000","last_up_change":"2023-01-25T05:37:39.272391+0000","last_in_change":"2023-01-25T05:37:32.512262+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T05:37:39.311339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T05:37:41.294509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e64ab012-13b0-40df-9047-e1cd4b1d9b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":92970},{"type":"v1","addr":"172.21.15.77:6818","nonce":92970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":92970},{"type":"v1","addr":"172.21.15.77:6822","nonce":92970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":92970},{"type":"v1","addr":"172.21.15.77:6830","nonce":92970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":92970},{"type":"v1","addr":"172.21.15.77:6826","nonce":92970}]},"public_addr":"172.21.15.77:6818/92970","cluster_addr":"172.21.15.77:6822/92970","heartbeat_back_addr":"172.21.15.77:6830/92970","heartbeat_front_addr":"172.21.15.77:6826/92970","state":["exists","up"]},{"osd":1,"uuid":"7e236477-7b67-43a9-8810-3c184b1e8613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":92968},{"type":"v1","addr":"172.21.15.77:6801","nonce":92968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":92968},{"type":"v1","addr":"172.21.15.77:6803","nonce":92968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":92968},{"type":"v1","addr":"172.21.15.77:6807","nonce":92968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":92968},{"type":"v1","addr":"172.21.15.77:6805","nonce":92968}]},"public_addr":"172.21.15.77:6801/92968","cluster_addr":"172.21.15.77:6803/92968","heartbeat_back_addr":"172.21.15.77:6807/92968","heartbeat_front_addr":"172.21.15.77:6805/92968","state":["exists","up"]},{"osd":2,"uuid":"4c7d5d87-fced-4c70-9ead-7b29cc05284d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":92969},{"type":"v1","addr":"172.21.15.77:6809","nonce":92969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":92969},{"type":"v1","addr":"172.21.15.77:6811","nonce":92969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":92969},{"type":"v1","addr":"172.21.15.77:6815","nonce":92969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":92969},{"type":"v1","addr":"172.21.15.77:6813","nonce":92969}]},"public_addr":"172.21.15.77:6809/92969","cluster_addr":"172.21.15.77:6811/92969","heartbeat_back_addr":"172.21.15.77:6815/92969","heartbeat_front_addr":"172.21.15.77:6813/92969","state":["exists","up"]},{"osd":3,"uuid":"4aff67d4-8f5d-4549-af8a-c815755ce937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6817","nonce":92971},{"type":"v1","addr":"172.21.15.77:6819","nonce":92971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6821","nonce":92971},{"type":"v1","addr":"172.21.15.77:6823","nonce":92971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6829","nonce":92971},{"type":"v1","addr":"172.21.15.77:6831","nonce":92971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6825","nonce":92971},{"type":"v1","addr":"172.21.15.77:6827","nonce":92971}]},"public_addr":"172.21.15.77:6819/92971","cluster_addr":"172.21.15.77:6823/92971","heartbeat_back_addr":"172.21.15.77:6831/92971","heartbeat_front_addr":"172.21.15.77:6827/92971","state":["exists","up"]},{"osd":4,"uuid":"b71752e2-4d8a-4c45-b193-ad8aa1fc6d87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":92931},{"type":"v1","addr":"172.21.15.121:6801","nonce":92931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":92931},{"type":"v1","addr":"172.21.15.121:6803","nonce":92931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":92931},{"type":"v1","addr":"172.21.15.121:6807","nonce":92931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":92931},{"type":"v1","addr":"172.21.15.121:6805","nonce":92931}]},"public_addr":"172.21.15.121:6801/92931","cluster_addr":"172.21.15.121:6803/92931","heartbeat_back_addr":"172.21.15.121:6807/92931","heartbeat_front_addr":"172.21.15.121:6805/92931","state":["exists","up"]},{"osd":5,"uuid":"6ad77028-3080-4e08-871b-d47b5744fe60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":92932},{"type":"v1","addr":"172.21.15.121:6810","nonce":92932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":92932},{"type":"v1","addr":"172.21.15.121:6814","nonce":92932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":92932},{"type":"v1","addr":"172.21.15.121:6821","nonce":92932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":92932},{"type":"v1","addr":"172.21.15.121:6818","nonce":92932}]},"public_addr":"172.21.15.121:6810/92932","cluster_addr":"172.21.15.121:6814/92932","heartbeat_back_addr":"172.21.15.121:6821/92932","heartbeat_front_addr":"172.21.15.121:6818/92932","state":["exists","up"]},{"osd":6,"uuid":"d2768cf4-a254-4169-919b-7fb789968ce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":92962},{"type":"v1","addr":"172.21.15.121:6825","nonce":92962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":92962},{"type":"v1","addr":"172.21.15.121:6827","nonce":92962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":92962},{"type":"v1","addr":"172.21.15.121:6831","nonce":92962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":92962},{"type":"v1","addr":"172.21.15.121:6829","nonce":92962}]},"public_addr":"172.21.15.121:6825/92962","cluster_addr":"172.21.15.121:6827/92962","heartbeat_back_addr":"172.21.15.121:6831/92962","heartbeat_front_addr":"172.21.15.121:6829/92962","state":["exists","up"]},{"osd":7,"uuid":"d5293edc-39b4-4fbe-88b6-1b1f4ca95390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6809","nonce":92933},{"type":"v1","addr":"172.21.15.121:6811","nonce":92933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6813","nonce":92933},{"type":"v1","addr":"172.21.15.121:6815","nonce":92933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":92933},{"type":"v1","addr":"172.21.15.121:6823","nonce":92933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6817","nonce":92933},{"type":"v1","addr":"172.21.15.121:6819","nonce":92933}]},"public_addr":"172.21.15.121:6811/92933","cluster_addr":"172.21.15.121:6815/92933","heartbeat_back_addr":"172.21.15.121:6823/92933","heartbeat_front_addr":"172.21.15.121:6819/92933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.744869+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.634040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.712891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.678441+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.707760+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.776908+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.783829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:37:37.749070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:37:43.421 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-25T05:37:43.421 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-25T05:37:43.422 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-25T05:37:43.423 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-25T05:37:43.423 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-25T05:37:43.424 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-25T05:37:43.424 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-25T05:37:43.425 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-25T05:37:43.702 INFO:teuthology.orchestra.run.smithi077.stdout:55834574851 2023-01-25T05:37:43.702 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T05:37:43.704 INFO:teuthology.orchestra.run.smithi077.stdout:55834574852 2023-01-25T05:37:43.704 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T05:37:43.707 INFO:teuthology.orchestra.run.smithi077.stdout:55834574851 2023-01-25T05:37:43.707 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T05:37:43.710 INFO:teuthology.orchestra.run.smithi077.stdout:55834574851 2023-01-25T05:37:43.711 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T05:37:43.719 INFO:teuthology.orchestra.run.smithi077.stdout:55834574851 2023-01-25T05:37:43.721 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T05:37:43.727 INFO:teuthology.orchestra.run.smithi077.stdout:55834574851 2023-01-25T05:37:43.728 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T05:37:43.737 INFO:teuthology.orchestra.run.smithi077.stdout:55834574852 2023-01-25T05:37:43.738 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T05:37:43.920 INFO:teuthology.orchestra.run.smithi077.stdout:55834574850 2023-01-25T05:37:43.921 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T05:37:44.285 INFO:teuthology.orchestra.run.smithi077.stdout:55834574851 2023-01-25T05:37:44.316 INFO:teuthology.orchestra.run.smithi077.stdout:55834574850 2023-01-25T05:37:44.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2023-01-25T05:37:44.345 INFO:teuthology.orchestra.run.smithi077.stdout:55834574850 2023-01-25T05:37:44.350 INFO:teuthology.orchestra.run.smithi077.stdout:55834574850 2023-01-25T05:37:44.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2023-01-25T05:37:44.364 INFO:teuthology.orchestra.run.smithi077.stdout:55834574850 2023-01-25T05:37:44.366 INFO:teuthology.orchestra.run.smithi077.stdout:55834574851 2023-01-25T05:37:44.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2023-01-25T05:37:44.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2023-01-25T05:37:44.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2023-01-25T05:37:44.391 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.7 2023-01-25T05:37:44.407 INFO:teuthology.orchestra.run.smithi077.stdout:55834574850 2023-01-25T05:37:44.422 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2023-01-25T05:37:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:55834574849 2023-01-25T05:37:44.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.6 2023-01-25T05:37:45.338 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T05:37:45.364 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T05:37:45.373 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T05:37:45.377 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T05:37:45.391 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T05:37:45.393 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T05:37:45.423 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T05:37:45.504 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T05:37:45.917 INFO:teuthology.orchestra.run.smithi077.stdout:55834574852 2023-01-25T05:37:45.937 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2023-01-25T05:37:45.937 DEBUG:teuthology.parallel:result is None 2023-01-25T05:37:45.964 INFO:teuthology.orchestra.run.smithi077.stdout:55834574851 2023-01-25T05:37:45.970 INFO:teuthology.orchestra.run.smithi077.stdout:55834574851 2023-01-25T05:37:45.974 INFO:teuthology.orchestra.run.smithi077.stdout:55834574852 2023-01-25T05:37:45.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2023-01-25T05:37:45.988 DEBUG:teuthology.parallel:result is None 2023-01-25T05:37:45.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.3 2023-01-25T05:37:45.996 DEBUG:teuthology.parallel:result is None 2023-01-25T05:37:45.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2023-01-25T05:37:45.998 DEBUG:teuthology.parallel:result is None 2023-01-25T05:37:46.022 INFO:teuthology.orchestra.run.smithi077.stdout:55834574851 2023-01-25T05:37:46.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2023-01-25T05:37:46.037 DEBUG:teuthology.parallel:result is None 2023-01-25T05:37:46.069 INFO:teuthology.orchestra.run.smithi077.stdout:55834574850 2023-01-25T05:37:46.076 INFO:teuthology.orchestra.run.smithi077.stdout:55834574851 2023-01-25T05:37:46.084 INFO:teuthology.orchestra.run.smithi077.stdout:55834574852 2023-01-25T05:37:46.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.6 2023-01-25T05:37:46.085 DEBUG:teuthology.parallel:result is None 2023-01-25T05:37:46.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2023-01-25T05:37:46.092 DEBUG:teuthology.parallel:result is None 2023-01-25T05:37:46.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.7 2023-01-25T05:37:46.099 DEBUG:teuthology.parallel:result is None 2023-01-25T05:37:46.099 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T05:37:46.101 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T05:37:46.453 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:37:46.457 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2023-01-25T05:37:46.466 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-25T05:37:45.262131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":84,"num_write":116,"num_write_kb":1375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41108,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":39680,"kb_avail":754933612,"statfs":{"total":773094113280,"available":773052018688,"internally_reserved":0,"allocated":1368064,"data_stored":1078576,"data_compressed":31516,"data_compressed_allocated":40960,"data_compressed_original":651264,"omap_allocated":0,"internal_metadata":40632320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.610375+0000","last_change":"2023-01-25T05:37:42.609174+0000","last_active":"2023-01-25T05:37:42.610375+0000","last_peered":"2023-01-25T05:37:42.610375+0000","last_clean":"2023-01-25T05:37:42.610375+0000","last_became_active":"2023-01-25T05:37:42.608795+0000","last_became_peered":"2023-01-25T05:37:42.608795+0000","last_unstale":"2023-01-25T05:37:42.610375+0000","last_undegraded":"2023-01-25T05:37:42.610375+0000","last_fullsized":"2023-01-25T05:37:42.610375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:06.941024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.316685+0000","last_change":"2023-01-25T05:37:42.605691+0000","last_active":"2023-01-25T05:37:43.316685+0000","last_peered":"2023-01-25T05:37:43.316685+0000","last_clean":"2023-01-25T05:37:43.316685+0000","last_became_active":"2023-01-25T05:37:42.605392+0000","last_became_peered":"2023-01-25T05:37:42.605392+0000","last_unstale":"2023-01-25T05:37:43.316685+0000","last_undegraded":"2023-01-25T05:37:43.316685+0000","last_fullsized":"2023-01-25T05:37:43.316685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:04.743126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.613116+0000","last_change":"2023-01-25T05:37:42.609576+0000","last_active":"2023-01-25T05:37:42.613116+0000","last_peered":"2023-01-25T05:37:42.613116+0000","last_clean":"2023-01-25T05:37:42.613116+0000","last_became_active":"2023-01-25T05:37:42.609210+0000","last_became_peered":"2023-01-25T05:37:42.609210+0000","last_unstale":"2023-01-25T05:37:42.613116+0000","last_undegraded":"2023-01-25T05:37:42.613116+0000","last_fullsized":"2023-01-25T05:37:42.613116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:07.363872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.609661+0000","last_change":"2023-01-25T05:37:42.609013+0000","last_active":"2023-01-25T05:37:42.609661+0000","last_peered":"2023-01-25T05:37:42.609661+0000","last_clean":"2023-01-25T05:37:42.609661+0000","last_became_active":"2023-01-25T05:37:42.608664+0000","last_became_peered":"2023-01-25T05:37:42.608664+0000","last_unstale":"2023-01-25T05:37:42.609661+0000","last_undegraded":"2023-01-25T05:37:42.609661+0000","last_fullsized":"2023-01-25T05:37:42.609661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:44.819351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'79","reported_seq":118,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.335171+0000","last_change":"2023-01-25T05:37:41.774655+0000","last_active":"2023-01-25T05:37:43.335171+0000","last_peered":"2023-01-25T05:37:43.335171+0000","last_clean":"2023-01-25T05:37:43.335171+0000","last_became_active":"2023-01-25T05:37:41.773570+0000","last_became_peered":"2023-01-25T05:37:41.773570+0000","last_unstale":"2023-01-25T05:37:43.335171+0000","last_undegraded":"2023-01-25T05:37:43.335171+0000","last_fullsized":"2023-01-25T05:37:43.335171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:40.275565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:40.275565+0000","last_clean_scrub_stamp":"2023-01-25T05:37:40.275565+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:56.165398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":84,"num_write":116,"num_write_kb":1375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.317018+0000","last_change":"2023-01-25T05:37:42.316404+0000","last_active":"2023-01-25T05:37:42.317018+0000","last_peered":"2023-01-25T05:37:42.317018+0000","last_clean":"2023-01-25T05:37:42.317018+0000","last_became_active":"2023-01-25T05:37:42.316020+0000","last_became_peered":"2023-01-25T05:37:42.316020+0000","last_unstale":"2023-01-25T05:37:42.317018+0000","last_undegraded":"2023-01-25T05:37:42.317018+0000","last_fullsized":"2023-01-25T05:37:42.317018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:02.243341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.318995+0000","last_change":"2023-01-25T05:37:42.311997+0000","last_active":"2023-01-25T05:37:42.318995+0000","last_peered":"2023-01-25T05:37:42.318995+0000","last_clean":"2023-01-25T05:37:42.318995+0000","last_became_active":"2023-01-25T05:37:42.311613+0000","last_became_peered":"2023-01-25T05:37:42.311613+0000","last_unstale":"2023-01-25T05:37:42.318995+0000","last_undegraded":"2023-01-25T05:37:42.318995+0000","last_fullsized":"2023-01-25T05:37:42.318995+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:52.659023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.609699+0000","last_change":"2023-01-25T05:37:42.609026+0000","last_active":"2023-01-25T05:37:42.609699+0000","last_peered":"2023-01-25T05:37:42.609699+0000","last_clean":"2023-01-25T05:37:42.609699+0000","last_became_active":"2023-01-25T05:37:42.608691+0000","last_became_peered":"2023-01-25T05:37:42.608691+0000","last_unstale":"2023-01-25T05:37:42.609699+0000","last_undegraded":"2023-01-25T05:37:42.609699+0000","last_fullsized":"2023-01-25T05:37:42.609699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:52.515428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.316389+0000","last_change":"2023-01-25T05:37:42.605094+0000","last_active":"2023-01-25T05:37:43.316389+0000","last_peered":"2023-01-25T05:37:43.316389+0000","last_clean":"2023-01-25T05:37:43.316389+0000","last_became_active":"2023-01-25T05:37:42.604745+0000","last_became_peered":"2023-01-25T05:37:42.604745+0000","last_unstale":"2023-01-25T05:37:43.316389+0000","last_undegraded":"2023-01-25T05:37:43.316389+0000","last_fullsized":"2023-01-25T05:37:43.316389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:59.539028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":84,"num_write":116,"num_write_kb":1375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1216548,"data_compressed":56358,"data_compressed_allocated":73728,"data_compressed_original":1167360,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":13,"seq":55834574850,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6030000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6520000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5180,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366660,"statfs":{"total":96636764160,"available":96631459840,"internally_reserved":0,"allocated":176128,"data_stored":186434,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":397.62299999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6609999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5412,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366428,"statfs":{"total":96636764160,"available":96631222272,"internally_reserved":0,"allocated":212992,"data_stored":458422,"data_compressed":18786,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5180,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366660,"statfs":{"total":96636764160,"available":96631459840,"internally_reserved":0,"allocated":176128,"data_stored":186434,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0140000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":405516,"data_compressed":18786,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":405516,"data_compressed":18786,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":405516,"data_compressed":18786,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T05:37:46.467 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T05:37:46.784 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:37:46.785 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2023-01-25T05:37:46.795 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-25T05:37:45.262131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":84,"num_write":116,"num_write_kb":1375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41108,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":39680,"kb_avail":754933612,"statfs":{"total":773094113280,"available":773052018688,"internally_reserved":0,"allocated":1368064,"data_stored":1078576,"data_compressed":31516,"data_compressed_allocated":40960,"data_compressed_original":651264,"omap_allocated":0,"internal_metadata":40632320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.610375+0000","last_change":"2023-01-25T05:37:42.609174+0000","last_active":"2023-01-25T05:37:42.610375+0000","last_peered":"2023-01-25T05:37:42.610375+0000","last_clean":"2023-01-25T05:37:42.610375+0000","last_became_active":"2023-01-25T05:37:42.608795+0000","last_became_peered":"2023-01-25T05:37:42.608795+0000","last_unstale":"2023-01-25T05:37:42.610375+0000","last_undegraded":"2023-01-25T05:37:42.610375+0000","last_fullsized":"2023-01-25T05:37:42.610375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:06.941024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.316685+0000","last_change":"2023-01-25T05:37:42.605691+0000","last_active":"2023-01-25T05:37:43.316685+0000","last_peered":"2023-01-25T05:37:43.316685+0000","last_clean":"2023-01-25T05:37:43.316685+0000","last_became_active":"2023-01-25T05:37:42.605392+0000","last_became_peered":"2023-01-25T05:37:42.605392+0000","last_unstale":"2023-01-25T05:37:43.316685+0000","last_undegraded":"2023-01-25T05:37:43.316685+0000","last_fullsized":"2023-01-25T05:37:43.316685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:04.743126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.613116+0000","last_change":"2023-01-25T05:37:42.609576+0000","last_active":"2023-01-25T05:37:42.613116+0000","last_peered":"2023-01-25T05:37:42.613116+0000","last_clean":"2023-01-25T05:37:42.613116+0000","last_became_active":"2023-01-25T05:37:42.609210+0000","last_became_peered":"2023-01-25T05:37:42.609210+0000","last_unstale":"2023-01-25T05:37:42.613116+0000","last_undegraded":"2023-01-25T05:37:42.613116+0000","last_fullsized":"2023-01-25T05:37:42.613116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:07.363872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.609661+0000","last_change":"2023-01-25T05:37:42.609013+0000","last_active":"2023-01-25T05:37:42.609661+0000","last_peered":"2023-01-25T05:37:42.609661+0000","last_clean":"2023-01-25T05:37:42.609661+0000","last_became_active":"2023-01-25T05:37:42.608664+0000","last_became_peered":"2023-01-25T05:37:42.608664+0000","last_unstale":"2023-01-25T05:37:42.609661+0000","last_undegraded":"2023-01-25T05:37:42.609661+0000","last_fullsized":"2023-01-25T05:37:42.609661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:44.819351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'79","reported_seq":118,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.335171+0000","last_change":"2023-01-25T05:37:41.774655+0000","last_active":"2023-01-25T05:37:43.335171+0000","last_peered":"2023-01-25T05:37:43.335171+0000","last_clean":"2023-01-25T05:37:43.335171+0000","last_became_active":"2023-01-25T05:37:41.773570+0000","last_became_peered":"2023-01-25T05:37:41.773570+0000","last_unstale":"2023-01-25T05:37:43.335171+0000","last_undegraded":"2023-01-25T05:37:43.335171+0000","last_fullsized":"2023-01-25T05:37:43.335171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:40.275565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:40.275565+0000","last_clean_scrub_stamp":"2023-01-25T05:37:40.275565+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:56.165398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":84,"num_write":116,"num_write_kb":1375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.317018+0000","last_change":"2023-01-25T05:37:42.316404+0000","last_active":"2023-01-25T05:37:42.317018+0000","last_peered":"2023-01-25T05:37:42.317018+0000","last_clean":"2023-01-25T05:37:42.317018+0000","last_became_active":"2023-01-25T05:37:42.316020+0000","last_became_peered":"2023-01-25T05:37:42.316020+0000","last_unstale":"2023-01-25T05:37:42.317018+0000","last_undegraded":"2023-01-25T05:37:42.317018+0000","last_fullsized":"2023-01-25T05:37:42.317018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:02.243341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.318995+0000","last_change":"2023-01-25T05:37:42.311997+0000","last_active":"2023-01-25T05:37:42.318995+0000","last_peered":"2023-01-25T05:37:42.318995+0000","last_clean":"2023-01-25T05:37:42.318995+0000","last_became_active":"2023-01-25T05:37:42.311613+0000","last_became_peered":"2023-01-25T05:37:42.311613+0000","last_unstale":"2023-01-25T05:37:42.318995+0000","last_undegraded":"2023-01-25T05:37:42.318995+0000","last_fullsized":"2023-01-25T05:37:42.318995+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:52.659023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.609699+0000","last_change":"2023-01-25T05:37:42.609026+0000","last_active":"2023-01-25T05:37:42.609699+0000","last_peered":"2023-01-25T05:37:42.609699+0000","last_clean":"2023-01-25T05:37:42.609699+0000","last_became_active":"2023-01-25T05:37:42.608691+0000","last_became_peered":"2023-01-25T05:37:42.608691+0000","last_unstale":"2023-01-25T05:37:42.609699+0000","last_undegraded":"2023-01-25T05:37:42.609699+0000","last_fullsized":"2023-01-25T05:37:42.609699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:52.515428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.316389+0000","last_change":"2023-01-25T05:37:42.605094+0000","last_active":"2023-01-25T05:37:43.316389+0000","last_peered":"2023-01-25T05:37:43.316389+0000","last_clean":"2023-01-25T05:37:43.316389+0000","last_became_active":"2023-01-25T05:37:42.604745+0000","last_became_peered":"2023-01-25T05:37:42.604745+0000","last_unstale":"2023-01-25T05:37:43.316389+0000","last_undegraded":"2023-01-25T05:37:43.316389+0000","last_fullsized":"2023-01-25T05:37:43.316389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:59.539028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":84,"num_write":116,"num_write_kb":1375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1216548,"data_compressed":56358,"data_compressed_allocated":73728,"data_compressed_original":1167360,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":13,"seq":55834574850,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6030000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6520000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5180,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366660,"statfs":{"total":96636764160,"available":96631459840,"internally_reserved":0,"allocated":176128,"data_stored":186434,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":397.62299999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6609999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5412,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366428,"statfs":{"total":96636764160,"available":96631222272,"internally_reserved":0,"allocated":212992,"data_stored":458422,"data_compressed":18786,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5180,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366660,"statfs":{"total":96636764160,"available":96631459840,"internally_reserved":0,"allocated":176128,"data_stored":186434,"data_compressed":6365,"data_compressed_allocated":8192,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0140000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":405516,"data_compressed":18786,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":405516,"data_compressed":18786,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":405516,"data_compressed":18786,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T05:37:46.798 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T05:37:46.798 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-25T05:37:46.799 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-25T05:37:46.799 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-25T05:37:47.161 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:37:47.161 INFO:teuthology.orchestra.run.smithi077.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-25T05:37:47.175 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-25T05:37:47.178 INFO:teuthology.run_tasks:Running task thrashosds... 2023-01-25T05:37:47.188 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-01-25T05:37:47.189 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri-testing-2023-01-23-0926'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri-testing-2023-01-23-0926', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}} 2023-01-25T05:37:47.189 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4} 2023-01-25T05:37:47.189 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-01-25T05:37:47.529 INFO:tasks.thrashosds:Beginning thrashosds... 2023-01-25T05:37:47.529 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T05:37:47.529 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T05:37:47.845 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:37:47.848 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2023-01-25T05:37:47.859 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-25T05:37:47.262503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41896,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":40320,"kb_avail":754932824,"statfs":{"total":773094113280,"available":773051211776,"internally_reserved":0,"allocated":1470464,"data_stored":1746496,"data_compressed":74079,"data_compressed_allocated":102400,"data_compressed_original":1290240,"omap_allocated":0,"internal_metadata":41287680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":20,"num_write":43,"num_write_kb":1041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000371"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.610375+0000","last_change":"2023-01-25T05:37:42.609174+0000","last_active":"2023-01-25T05:37:42.610375+0000","last_peered":"2023-01-25T05:37:42.610375+0000","last_clean":"2023-01-25T05:37:42.610375+0000","last_became_active":"2023-01-25T05:37:42.608795+0000","last_became_peered":"2023-01-25T05:37:42.608795+0000","last_unstale":"2023-01-25T05:37:42.610375+0000","last_undegraded":"2023-01-25T05:37:42.610375+0000","last_fullsized":"2023-01-25T05:37:42.610375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:06.941024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.316685+0000","last_change":"2023-01-25T05:37:42.605691+0000","last_active":"2023-01-25T05:37:43.316685+0000","last_peered":"2023-01-25T05:37:43.316685+0000","last_clean":"2023-01-25T05:37:43.316685+0000","last_became_active":"2023-01-25T05:37:42.605392+0000","last_became_peered":"2023-01-25T05:37:42.605392+0000","last_unstale":"2023-01-25T05:37:43.316685+0000","last_undegraded":"2023-01-25T05:37:43.316685+0000","last_fullsized":"2023-01-25T05:37:43.316685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:04.743126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.613116+0000","last_change":"2023-01-25T05:37:42.609576+0000","last_active":"2023-01-25T05:37:42.613116+0000","last_peered":"2023-01-25T05:37:42.613116+0000","last_clean":"2023-01-25T05:37:42.613116+0000","last_became_active":"2023-01-25T05:37:42.609210+0000","last_became_peered":"2023-01-25T05:37:42.609210+0000","last_unstale":"2023-01-25T05:37:42.613116+0000","last_undegraded":"2023-01-25T05:37:42.613116+0000","last_fullsized":"2023-01-25T05:37:42.613116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:07.363872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.609661+0000","last_change":"2023-01-25T05:37:42.609013+0000","last_active":"2023-01-25T05:37:42.609661+0000","last_peered":"2023-01-25T05:37:42.609661+0000","last_clean":"2023-01-25T05:37:42.609661+0000","last_became_active":"2023-01-25T05:37:42.608664+0000","last_became_peered":"2023-01-25T05:37:42.608664+0000","last_unstale":"2023-01-25T05:37:42.609661+0000","last_undegraded":"2023-01-25T05:37:42.609661+0000","last_fullsized":"2023-01-25T05:37:42.609661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:44.819351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:45.115216+0000","last_change":"2023-01-25T05:37:41.774655+0000","last_active":"2023-01-25T05:37:45.115216+0000","last_peered":"2023-01-25T05:37:45.115216+0000","last_clean":"2023-01-25T05:37:45.115216+0000","last_became_active":"2023-01-25T05:37:41.773570+0000","last_became_peered":"2023-01-25T05:37:41.773570+0000","last_unstale":"2023-01-25T05:37:45.115216+0000","last_undegraded":"2023-01-25T05:37:45.115216+0000","last_fullsized":"2023-01-25T05:37:45.115216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:40.275565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:40.275565+0000","last_clean_scrub_stamp":"2023-01-25T05:37:40.275565+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:56.165398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.317018+0000","last_change":"2023-01-25T05:37:42.316404+0000","last_active":"2023-01-25T05:37:42.317018+0000","last_peered":"2023-01-25T05:37:42.317018+0000","last_clean":"2023-01-25T05:37:42.317018+0000","last_became_active":"2023-01-25T05:37:42.316020+0000","last_became_peered":"2023-01-25T05:37:42.316020+0000","last_unstale":"2023-01-25T05:37:42.317018+0000","last_undegraded":"2023-01-25T05:37:42.317018+0000","last_fullsized":"2023-01-25T05:37:42.317018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:02.243341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.719322+0000","last_change":"2023-01-25T05:37:42.311997+0000","last_active":"2023-01-25T05:37:43.719322+0000","last_peered":"2023-01-25T05:37:43.719322+0000","last_clean":"2023-01-25T05:37:43.719322+0000","last_became_active":"2023-01-25T05:37:42.311613+0000","last_became_peered":"2023-01-25T05:37:42.311613+0000","last_unstale":"2023-01-25T05:37:43.719322+0000","last_undegraded":"2023-01-25T05:37:43.719322+0000","last_fullsized":"2023-01-25T05:37:43.719322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:52.659023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.609699+0000","last_change":"2023-01-25T05:37:42.609026+0000","last_active":"2023-01-25T05:37:42.609699+0000","last_peered":"2023-01-25T05:37:42.609699+0000","last_clean":"2023-01-25T05:37:42.609699+0000","last_became_active":"2023-01-25T05:37:42.608691+0000","last_became_peered":"2023-01-25T05:37:42.608691+0000","last_unstale":"2023-01-25T05:37:42.609699+0000","last_undegraded":"2023-01-25T05:37:42.609699+0000","last_fullsized":"2023-01-25T05:37:42.609699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:52.515428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.316389+0000","last_change":"2023-01-25T05:37:42.605094+0000","last_active":"2023-01-25T05:37:43.316389+0000","last_peered":"2023-01-25T05:37:43.316389+0000","last_clean":"2023-01-25T05:37:43.316389+0000","last_became_active":"2023-01-25T05:37:42.604745+0000","last_became_peered":"2023-01-25T05:37:42.604745+0000","last_unstale":"2023-01-25T05:37:43.316389+0000","last_undegraded":"2023-01-25T05:37:43.316389+0000","last_fullsized":"2023-01-25T05:37:43.316389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:59.539028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":1586784,"data_compressed":88272,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6030000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6520000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5564,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94366276,"statfs":{"total":96636764160,"available":96631066624,"internally_reserved":0,"allocated":225280,"data_stored":520394,"data_compressed":26464,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":397.62299999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6609999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5432,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366408,"statfs":{"total":96636764160,"available":96631201792,"internally_reserved":0,"allocated":217088,"data_stored":458422,"data_compressed":21151,"data_compressed_allocated":28672,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5564,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94366276,"statfs":{"total":96636764160,"available":96631066624,"internally_reserved":0,"allocated":225280,"data_stored":520394,"data_compressed":26464,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0140000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29424,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29424,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29424,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T05:37:47.861 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T05:37:48.178 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-01-25T05:37:48.180 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2023-01-25T05:37:48.191 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-25T05:37:47.262503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41896,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":40320,"kb_avail":754932824,"statfs":{"total":773094113280,"available":773051211776,"internally_reserved":0,"allocated":1470464,"data_stored":1746496,"data_compressed":74079,"data_compressed_allocated":102400,"data_compressed_original":1290240,"omap_allocated":0,"internal_metadata":41287680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":20,"num_write":43,"num_write_kb":1041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000371"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.610375+0000","last_change":"2023-01-25T05:37:42.609174+0000","last_active":"2023-01-25T05:37:42.610375+0000","last_peered":"2023-01-25T05:37:42.610375+0000","last_clean":"2023-01-25T05:37:42.610375+0000","last_became_active":"2023-01-25T05:37:42.608795+0000","last_became_peered":"2023-01-25T05:37:42.608795+0000","last_unstale":"2023-01-25T05:37:42.610375+0000","last_undegraded":"2023-01-25T05:37:42.610375+0000","last_fullsized":"2023-01-25T05:37:42.610375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:06.941024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.316685+0000","last_change":"2023-01-25T05:37:42.605691+0000","last_active":"2023-01-25T05:37:43.316685+0000","last_peered":"2023-01-25T05:37:43.316685+0000","last_clean":"2023-01-25T05:37:43.316685+0000","last_became_active":"2023-01-25T05:37:42.605392+0000","last_became_peered":"2023-01-25T05:37:42.605392+0000","last_unstale":"2023-01-25T05:37:43.316685+0000","last_undegraded":"2023-01-25T05:37:43.316685+0000","last_fullsized":"2023-01-25T05:37:43.316685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:04.743126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.613116+0000","last_change":"2023-01-25T05:37:42.609576+0000","last_active":"2023-01-25T05:37:42.613116+0000","last_peered":"2023-01-25T05:37:42.613116+0000","last_clean":"2023-01-25T05:37:42.613116+0000","last_became_active":"2023-01-25T05:37:42.609210+0000","last_became_peered":"2023-01-25T05:37:42.609210+0000","last_unstale":"2023-01-25T05:37:42.613116+0000","last_undegraded":"2023-01-25T05:37:42.613116+0000","last_fullsized":"2023-01-25T05:37:42.613116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:07.363872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.609661+0000","last_change":"2023-01-25T05:37:42.609013+0000","last_active":"2023-01-25T05:37:42.609661+0000","last_peered":"2023-01-25T05:37:42.609661+0000","last_clean":"2023-01-25T05:37:42.609661+0000","last_became_active":"2023-01-25T05:37:42.608664+0000","last_became_peered":"2023-01-25T05:37:42.608664+0000","last_unstale":"2023-01-25T05:37:42.609661+0000","last_undegraded":"2023-01-25T05:37:42.609661+0000","last_fullsized":"2023-01-25T05:37:42.609661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:44.819351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:45.115216+0000","last_change":"2023-01-25T05:37:41.774655+0000","last_active":"2023-01-25T05:37:45.115216+0000","last_peered":"2023-01-25T05:37:45.115216+0000","last_clean":"2023-01-25T05:37:45.115216+0000","last_became_active":"2023-01-25T05:37:41.773570+0000","last_became_peered":"2023-01-25T05:37:41.773570+0000","last_unstale":"2023-01-25T05:37:45.115216+0000","last_undegraded":"2023-01-25T05:37:45.115216+0000","last_fullsized":"2023-01-25T05:37:45.115216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:40.275565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:40.275565+0000","last_clean_scrub_stamp":"2023-01-25T05:37:40.275565+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:56.165398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.317018+0000","last_change":"2023-01-25T05:37:42.316404+0000","last_active":"2023-01-25T05:37:42.317018+0000","last_peered":"2023-01-25T05:37:42.317018+0000","last_clean":"2023-01-25T05:37:42.317018+0000","last_became_active":"2023-01-25T05:37:42.316020+0000","last_became_peered":"2023-01-25T05:37:42.316020+0000","last_unstale":"2023-01-25T05:37:42.317018+0000","last_undegraded":"2023-01-25T05:37:42.317018+0000","last_fullsized":"2023-01-25T05:37:42.317018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:39:02.243341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.719322+0000","last_change":"2023-01-25T05:37:42.311997+0000","last_active":"2023-01-25T05:37:43.719322+0000","last_peered":"2023-01-25T05:37:43.719322+0000","last_clean":"2023-01-25T05:37:43.719322+0000","last_became_active":"2023-01-25T05:37:42.311613+0000","last_became_peered":"2023-01-25T05:37:42.311613+0000","last_unstale":"2023-01-25T05:37:43.719322+0000","last_undegraded":"2023-01-25T05:37:43.719322+0000","last_fullsized":"2023-01-25T05:37:43.719322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:52.659023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T05:37:42.609699+0000","last_change":"2023-01-25T05:37:42.609026+0000","last_active":"2023-01-25T05:37:42.609699+0000","last_peered":"2023-01-25T05:37:42.609699+0000","last_clean":"2023-01-25T05:37:42.609699+0000","last_became_active":"2023-01-25T05:37:42.608691+0000","last_became_peered":"2023-01-25T05:37:42.608691+0000","last_unstale":"2023-01-25T05:37:42.609699+0000","last_undegraded":"2023-01-25T05:37:42.609699+0000","last_fullsized":"2023-01-25T05:37:42.609699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:52.515428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T05:37:43.316389+0000","last_change":"2023-01-25T05:37:42.605094+0000","last_active":"2023-01-25T05:37:43.316389+0000","last_peered":"2023-01-25T05:37:43.316389+0000","last_clean":"2023-01-25T05:37:43.316389+0000","last_became_active":"2023-01-25T05:37:42.604745+0000","last_became_peered":"2023-01-25T05:37:42.604745+0000","last_unstale":"2023-01-25T05:37:43.316389+0000","last_undegraded":"2023-01-25T05:37:43.316389+0000","last_fullsized":"2023-01-25T05:37:43.316389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:37:41.294571+0000","last_clean_scrub_stamp":"2023-01-25T05:37:41.294571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:38:59.539028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":104,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":1586784,"data_compressed":88272,"data_compressed_allocated":122880,"data_compressed_original":1536000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6030000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6520000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5564,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94366276,"statfs":{"total":96636764160,"available":96631066624,"internally_reserved":0,"allocated":225280,"data_stored":520394,"data_compressed":26464,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":397.62299999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6609999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5432,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94366408,"statfs":{"total":96636764160,"available":96631201792,"internally_reserved":0,"allocated":217088,"data_stored":458422,"data_compressed":21151,"data_compressed_allocated":28672,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":47158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5564,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94366276,"statfs":{"total":96636764160,"available":96631066624,"internally_reserved":0,"allocated":225280,"data_stored":520394,"data_compressed":26464,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0140000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29424,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29424,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29424,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T05:37:48.193 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T05:37:48.193 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-01-25T05:37:48.508 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 17 2023-01-25T05:37:48.509 INFO:teuthology.orchestra.run.smithi077.stdout:fsid 7bfb8493-ac34-42fb-8797-f07adda855bd 2023-01-25T05:37:48.509 INFO:teuthology.orchestra.run.smithi077.stdout:created 2023-01-25T05:37:29.945290+0000 2023-01-25T05:37:48.509 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2023-01-25T05:37:43.305128+0000 2023-01-25T05:37:48.509 INFO:teuthology.orchestra.run.smithi077.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-01-25T05:37:48.509 INFO:teuthology.orchestra.run.smithi077.stdout:crush_version 4 2023-01-25T05:37:48.510 INFO:teuthology.orchestra.run.smithi077.stdout:full_ratio 0.95 2023-01-25T05:37:48.510 INFO:teuthology.orchestra.run.smithi077.stdout:backfillfull_ratio 0.9 2023-01-25T05:37:48.510 INFO:teuthology.orchestra.run.smithi077.stdout:nearfull_ratio 0.85 2023-01-25T05:37:48.510 INFO:teuthology.orchestra.run.smithi077.stdout:require_min_compat_client luminous 2023-01-25T05:37:48.510 INFO:teuthology.orchestra.run.smithi077.stdout:min_compat_client jewel 2023-01-25T05:37:48.511 INFO:teuthology.orchestra.run.smithi077.stdout:require_osd_release reef 2023-01-25T05:37:48.511 INFO:teuthology.orchestra.run.smithi077.stdout:stretch_mode_enabled false 2023-01-25T05:37:48.511 INFO:teuthology.orchestra.run.smithi077.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2023-01-25T05:37:48.511 INFO:teuthology.orchestra.run.smithi077.stdout:pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 application rbd 2023-01-25T05:37:48.511 INFO:teuthology.orchestra.run.smithi077.stdout:max_osd 8 2023-01-25T05:37:48.512 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6816/92970,v1:172.21.15.77:6818/92970] [v2:172.21.15.77:6820/92970,v1:172.21.15.77:6822/92970] exists,up e64ab012-13b0-40df-9047-e1cd4b1d9b30 2023-01-25T05:37:48.512 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6800/92968,v1:172.21.15.77:6801/92968] [v2:172.21.15.77:6802/92968,v1:172.21.15.77:6803/92968] exists,up 7e236477-7b67-43a9-8810-3c184b1e8613 2023-01-25T05:37:48.512 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6808/92969,v1:172.21.15.77:6809/92969] [v2:172.21.15.77:6810/92969,v1:172.21.15.77:6811/92969] exists,up 4c7d5d87-fced-4c70-9ead-7b29cc05284d 2023-01-25T05:37:48.512 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6817/92971,v1:172.21.15.77:6819/92971] [v2:172.21.15.77:6821/92971,v1:172.21.15.77:6823/92971] exists,up 4aff67d4-8f5d-4549-af8a-c815755ce937 2023-01-25T05:37:48.512 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6800/92931,v1:172.21.15.121:6801/92931] [v2:172.21.15.121:6802/92931,v1:172.21.15.121:6803/92931] exists,up b71752e2-4d8a-4c45-b193-ad8aa1fc6d87 2023-01-25T05:37:48.513 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6808/92932,v1:172.21.15.121:6810/92932] [v2:172.21.15.121:6812/92932,v1:172.21.15.121:6814/92932] exists,up 6ad77028-3080-4e08-871b-d47b5744fe60 2023-01-25T05:37:48.513 INFO:teuthology.orchestra.run.smithi077.stdout:osd.6 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6824/92962,v1:172.21.15.121:6825/92962] [v2:172.21.15.121:6826/92962,v1:172.21.15.121:6827/92962] exists,up d2768cf4-a254-4169-919b-7fb789968ce7 2023-01-25T05:37:48.513 INFO:teuthology.orchestra.run.smithi077.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6809/92933,v1:172.21.15.121:6811/92933] [v2:172.21.15.121:6813/92933,v1:172.21.15.121:6815/92933] exists,up d5293edc-39b4-4fbe-88b6-1b1f4ca95390 2023-01-25T05:37:48.522 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6816/92970,v1:172.21.15.77:6818/92970] [v2:172.21.15.77:6820/92970,v1:172.21.15.77:6822/92970] exists,up e64ab012-13b0-40df-9047-e1cd4b1d9b30', 'osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6800/92968,v1:172.21.15.77:6801/92968] [v2:172.21.15.77:6802/92968,v1:172.21.15.77:6803/92968] exists,up 7e236477-7b67-43a9-8810-3c184b1e8613', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6808/92969,v1:172.21.15.77:6809/92969] [v2:172.21.15.77:6810/92969,v1:172.21.15.77:6811/92969] exists,up 4c7d5d87-fced-4c70-9ead-7b29cc05284d', 'osd.3 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6817/92971,v1:172.21.15.77:6819/92971] [v2:172.21.15.77:6821/92971,v1:172.21.15.77:6823/92971] exists,up 4aff67d4-8f5d-4549-af8a-c815755ce937', 'osd.4 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6800/92931,v1:172.21.15.121:6801/92931] [v2:172.21.15.121:6802/92931,v1:172.21.15.121:6803/92931] exists,up b71752e2-4d8a-4c45-b193-ad8aa1fc6d87', 'osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6808/92932,v1:172.21.15.121:6810/92932] [v2:172.21.15.121:6812/92932,v1:172.21.15.121:6814/92932] exists,up 6ad77028-3080-4e08-871b-d47b5744fe60', 'osd.6 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6824/92962,v1:172.21.15.121:6825/92962] [v2:172.21.15.121:6826/92962,v1:172.21.15.121:6827/92962] exists,up d2768cf4-a254-4169-919b-7fb789968ce7', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6809/92933,v1:172.21.15.121:6811/92933] [v2:172.21.15.121:6813/92933,v1:172.21.15.121:6815/92933] exists,up d5293edc-39b4-4fbe-88b6-1b1f4ca95390'] 2023-01-25T05:37:48.523 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-01-25T05:37:48.643 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-01-25T05:37:48.643 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mon.a", 2023-01-25T05:37:48.643 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster": "ceph", 2023-01-25T05:37:48.644 INFO:teuthology.orchestra.run.smithi077.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-01-25T05:37:48.644 INFO:teuthology.orchestra.run.smithi077.stdout: "admin_socket_mode": "", 2023-01-25T05:37:48.644 INFO:teuthology.orchestra.run.smithi077.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-01-25T05:37:48.644 INFO:teuthology.orchestra.run.smithi077.stdout: "auth_client_required": "cephx, none", 2023-01-25T05:37:48.644 INFO:teuthology.orchestra.run.smithi077.stdout: "auth_cluster_required": "cephx", 2023-01-25T05:37:48.645 INFO:teuthology.orchestra.run.smithi077.stdout: "auth_debug": "true", 2023-01-25T05:37:48.645 INFO:teuthology.orchestra.run.smithi077.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-01-25T05:37:48.645 INFO:teuthology.orchestra.run.smithi077.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-01-25T05:37:48.645 INFO:teuthology.orchestra.run.smithi077.stdout: "auth_service_required": "cephx", 2023-01-25T05:37:48.645 INFO:teuthology.orchestra.run.smithi077.stdout: "auth_service_ticket_ttl": "240.000000", 2023-01-25T05:37:48.646 INFO:teuthology.orchestra.run.smithi077.stdout: "auth_supported": "cephx", 2023-01-25T05:37:48.646 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_aio": "true", 2023-01-25T05:37:48.646 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_aio_max_queue_depth": "1024", 2023-01-25T05:37:48.646 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_aio_poll_ms": "250", 2023-01-25T05:37:48.646 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_aio_reap_max": "16", 2023-01-25T05:37:48.647 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_async_discard": "false", 2023-01-25T05:37:48.647 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_block_size": "4096", 2023-01-25T05:37:48.647 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_debug_aio": "false", 2023-01-25T05:37:48.647 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-01-25T05:37:48.647 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-01-25T05:37:48.648 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_debug_inflight_ios": "false", 2023-01-25T05:37:48.648 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_enable_discard": "false", 2023-01-25T05:37:48.648 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_flock_retry": "3", 2023-01-25T05:37:48.648 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_flock_retry_interval": "0.100000", 2023-01-25T05:37:48.648 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_inject_crash": "0", 2023-01-25T05:37:48.649 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_inject_crash_flush_delay": "2", 2023-01-25T05:37:48.649 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_ioring": "false", 2023-01-25T05:37:48.649 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_ioring_hipri": "false", 2023-01-25T05:37:48.649 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_ioring_sqthread_poll": "false", 2023-01-25T05:37:48.649 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-01-25T05:37:48.650 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_read_buffer_alignment": "4096", 2023-01-25T05:37:48.650 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-01-25T05:37:48.650 INFO:teuthology.orchestra.run.smithi077.stdout: "bdev_type": "", 2023-01-25T05:37:48.650 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_alloc_size": "1048576", 2023-01-25T05:37:48.650 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_allocator": "hybrid", 2023-01-25T05:37:48.651 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_buffered_io": "true", 2023-01-25T05:37:48.651 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_check_for_zeros": "false", 2023-01-25T05:37:48.651 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_check_volume_selector_often": "false", 2023-01-25T05:37:48.651 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-01-25T05:37:48.651 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_compact_log_sync": "false", 2023-01-25T05:37:48.651 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-01-25T05:37:48.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_log_compact_min_size": "16777216", 2023-01-25T05:37:48.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_log_replay_check_allocations": "true", 2023-01-25T05:37:48.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_max_log_runway": "4194304", 2023-01-25T05:37:48.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_max_prefetch": "1048576", 2023-01-25T05:37:48.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_min_flush_size": "524288", 2023-01-25T05:37:48.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_min_log_runway": "1048576", 2023-01-25T05:37:48.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_replay_recovery": "false", 2023-01-25T05:37:48.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-01-25T05:37:48.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_shared_alloc_size": "65536", 2023-01-25T05:37:48.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_sync_write": "false", 2023-01-25T05:37:48.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-01-25T05:37:48.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-01-25T05:37:48.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-01-25T05:37:48.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_allocation_from_file": "true", 2023-01-25T05:37:48.655 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_allocator": "hybrid", 2023-01-25T05:37:48.655 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-01-25T05:37:48.655 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-01-25T05:37:48.655 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-01-25T05:37:48.655 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-01-25T05:37:48.656 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-01-25T05:37:48.656 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-01-25T05:37:48.656 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blobid_prealloc": "10240", 2023-01-25T05:37:48.656 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_block_create": "true", 2023-01-25T05:37:48.656 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_block_db_create": "false", 2023-01-25T05:37:48.657 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_block_db_path": "", 2023-01-25T05:37:48.657 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_block_db_size": "0", 2023-01-25T05:37:48.657 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_block_path": "", 2023-01-25T05:37:48.657 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_block_preallocate_file": "false", 2023-01-25T05:37:48.657 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_block_size": "107374182400", 2023-01-25T05:37:48.658 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_block_wal_create": "false", 2023-01-25T05:37:48.658 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_block_wal_path": "", 2023-01-25T05:37:48.658 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_block_wal_size": "100663296", 2023-01-25T05:37:48.658 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_bluefs": "true", 2023-01-25T05:37:48.658 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-01-25T05:37:48.658 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_bluefs_env_mirror": "false", 2023-01-25T05:37:48.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-01-25T05:37:48.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-01-25T05:37:48.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T05:37:48.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T05:37:48.659 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-01-25T05:37:48.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T05:37:48.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_autotune": "true", 2023-01-25T05:37:48.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-01-25T05:37:48.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-01-25T05:37:48.660 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-01-25T05:37:48.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-01-25T05:37:48.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_size": "0", 2023-01-25T05:37:48.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-01-25T05:37:48.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-01-25T05:37:48.661 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-01-25T05:37:48.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-01-25T05:37:48.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_type": "2q", 2023-01-25T05:37:48.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-01-25T05:37:48.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_clone_cow": "true", 2023-01-25T05:37:48.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_compression_algorithm": "snappy", 2023-01-25T05:37:48.662 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_compression_max_blob_size": "0", 2023-01-25T05:37:48.663 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-01-25T05:37:48.663 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-01-25T05:37:48.663 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_compression_min_blob_size": "0", 2023-01-25T05:37:48.663 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-01-25T05:37:48.663 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-01-25T05:37:48.664 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_compression_mode": "none", 2023-01-25T05:37:48.664 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-01-25T05:37:48.664 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_csum_type": "crc32c", 2023-01-25T05:37:48.664 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_enforce_settings": "default", 2023-01-25T05:37:48.664 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_freelist": "false", 2023-01-25T05:37:48.665 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_fsck_abort": "false", 2023-01-25T05:37:48.665 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2023-01-25T05:37:48.665 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_inject_bug21040": "false", 2023-01-25T05:37:48.665 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-01-25T05:37:48.665 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_inject_read_err": "false", 2023-01-25T05:37:48.666 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_legacy_omap": "false", 2023-01-25T05:37:48.666 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-01-25T05:37:48.666 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-01-25T05:37:48.666 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-01-25T05:37:48.666 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-01-25T05:37:48.667 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_prefill": "0.000000", 2023-01-25T05:37:48.667 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-01-25T05:37:48.667 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-01-25T05:37:48.667 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-01-25T05:37:48.667 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_small_allocations": "0", 2023-01-25T05:37:48.668 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-01-25T05:37:48.668 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_default_buffered_read": "true", 2023-01-25T05:37:48.668 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_default_buffered_write": "false", 2023-01-25T05:37:48.668 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_deferred_batch_ops": "0", 2023-01-25T05:37:48.668 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-01-25T05:37:48.668 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-01-25T05:37:48.669 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-01-25T05:37:48.669 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-01-25T05:37:48.669 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-01-25T05:37:48.669 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-01-25T05:37:48.669 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-01-25T05:37:48.670 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fail_eio": "false", 2023-01-25T05:37:48.670 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-01-25T05:37:48.670 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-01-25T05:37:48.670 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-01-25T05:37:48.670 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-01-25T05:37:48.671 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_on_mkfs": "true", 2023-01-25T05:37:48.671 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-01-25T05:37:48.671 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_on_mount": "false", 2023-01-25T05:37:48.671 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-01-25T05:37:48.671 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_on_umount": "false", 2023-01-25T05:37:48.671 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-01-25T05:37:48.672 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-01-25T05:37:48.672 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-01-25T05:37:48.672 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-01-25T05:37:48.672 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-01-25T05:37:48.672 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-01-25T05:37:48.673 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-01-25T05:37:48.673 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-01-25T05:37:48.673 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_ignore_data_csum": "false", 2023-01-25T05:37:48.673 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-01-25T05:37:48.673 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_kvbackend": "rocksdb", 2023-01-25T05:37:48.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-01-25T05:37:48.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-01-25T05:37:48.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_log_op_age": "5.000000", 2023-01-25T05:37:48.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_max_alloc_size": "0", 2023-01-25T05:37:48.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_max_blob_size": "0", 2023-01-25T05:37:48.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-01-25T05:37:48.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-01-25T05:37:48.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_max_defer_interval": "3.000000", 2023-01-25T05:37:48.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_max_deferred_txc": "32", 2023-01-25T05:37:48.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_min_alloc_size": "0", 2023-01-25T05:37:48.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-01-25T05:37:48.676 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-01-25T05:37:48.676 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_nid_prealloc": "1024", 2023-01-25T05:37:48.676 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_prefer_deferred_size": "0", 2023-01-25T05:37:48.676 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-01-25T05:37:48.676 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-01-25T05:37:48.677 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_qfsck_on_mount": "true", 2023-01-25T05:37:48.677 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_retry_disk_reads": "3", 2023-01-25T05:37:48.677 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_rocksdb_cf": "true", 2023-01-25T05:37:48.677 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2023-01-25T05:37:48.677 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=128,min_write_buffer_number_to_merge=16,compaction_style=kCompactionStyleLevel,write_buffer_size=8388608,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0,ttl=21600", 2023-01-25T05:37:48.678 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_rocksdb_options_annex": "", 2023-01-25T05:37:48.678 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_spdk_coremask": "0x1", 2023-01-25T05:37:48.678 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_spdk_io_sleep": "5", 2023-01-25T05:37:48.678 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_spdk_max_io_completion": "0", 2023-01-25T05:37:48.678 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_spdk_mem": "512", 2023-01-25T05:37:48.679 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_sync_submit_transaction": "false", 2023-01-25T05:37:48.679 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_throttle_bytes": "67108864", 2023-01-25T05:37:48.679 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_throttle_cost_per_io": "0", 2023-01-25T05:37:48.679 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-01-25T05:37:48.679 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-01-25T05:37:48.680 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-01-25T05:37:48.680 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-01-25T05:37:48.680 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_tracing": "false", 2023-01-25T05:37:48.680 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-01-25T05:37:48.680 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-01-25T05:37:48.681 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_volume_selection_reserved": "0", 2023-01-25T05:37:48.681 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-01-25T05:37:48.681 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-01-25T05:37:48.681 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-01-25T05:37:48.681 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-01-25T05:37:48.682 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-01-25T05:37:48.682 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-01-25T05:37:48.682 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_zero_block_detection": "false", 2023-01-25T05:37:48.682 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-01-25T05:37:48.682 INFO:teuthology.orchestra.run.smithi077.stdout: "cephfs_mirror_action_update_interval": "2", 2023-01-25T05:37:48.683 INFO:teuthology.orchestra.run.smithi077.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-01-25T05:37:48.683 INFO:teuthology.orchestra.run.smithi077.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-01-25T05:37:48.683 INFO:teuthology.orchestra.run.smithi077.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-01-25T05:37:48.683 INFO:teuthology.orchestra.run.smithi077.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-01-25T05:37:48.683 INFO:teuthology.orchestra.run.smithi077.stdout: "cephfs_mirror_mount_timeout": "10", 2023-01-25T05:37:48.684 INFO:teuthology.orchestra.run.smithi077.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-01-25T05:37:48.684 INFO:teuthology.orchestra.run.smithi077.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-01-25T05:37:48.684 INFO:teuthology.orchestra.run.smithi077.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-01-25T05:37:48.684 INFO:teuthology.orchestra.run.smithi077.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-01-25T05:37:48.684 INFO:teuthology.orchestra.run.smithi077.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-01-25T05:37:48.685 INFO:teuthology.orchestra.run.smithi077.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-01-25T05:37:48.685 INFO:teuthology.orchestra.run.smithi077.stdout: "cephx_cluster_require_signatures": "false", 2023-01-25T05:37:48.685 INFO:teuthology.orchestra.run.smithi077.stdout: "cephx_cluster_require_version": "2", 2023-01-25T05:37:48.685 INFO:teuthology.orchestra.run.smithi077.stdout: "cephx_require_signatures": "false", 2023-01-25T05:37:48.685 INFO:teuthology.orchestra.run.smithi077.stdout: "cephx_require_version": "2", 2023-01-25T05:37:48.685 INFO:teuthology.orchestra.run.smithi077.stdout: "cephx_service_require_signatures": "false", 2023-01-25T05:37:48.686 INFO:teuthology.orchestra.run.smithi077.stdout: "cephx_service_require_version": "2", 2023-01-25T05:37:48.686 INFO:teuthology.orchestra.run.smithi077.stdout: "cephx_sign_messages": "true", 2023-01-25T05:37:48.686 INFO:teuthology.orchestra.run.smithi077.stdout: "chdir": "", 2023-01-25T05:37:48.686 INFO:teuthology.orchestra.run.smithi077.stdout: "client_acl_type": "", 2023-01-25T05:37:48.686 INFO:teuthology.orchestra.run.smithi077.stdout: "client_asio_thread_count": "2", 2023-01-25T05:37:48.687 INFO:teuthology.orchestra.run.smithi077.stdout: "client_cache_mid": "0.750000", 2023-01-25T05:37:48.687 INFO:teuthology.orchestra.run.smithi077.stdout: "client_cache_size": "16384", 2023-01-25T05:37:48.687 INFO:teuthology.orchestra.run.smithi077.stdout: "client_caps_release_delay": "5", 2023-01-25T05:37:48.687 INFO:teuthology.orchestra.run.smithi077.stdout: "client_check_pool_perm": "true", 2023-01-25T05:37:48.687 INFO:teuthology.orchestra.run.smithi077.stdout: "client_collect_and_send_global_metrics": "false", 2023-01-25T05:37:48.688 INFO:teuthology.orchestra.run.smithi077.stdout: "client_debug_force_sync_read": "false", 2023-01-25T05:37:48.688 INFO:teuthology.orchestra.run.smithi077.stdout: "client_debug_getattr_caps": "false", 2023-01-25T05:37:48.688 INFO:teuthology.orchestra.run.smithi077.stdout: "client_debug_inject_tick_delay": "0", 2023-01-25T05:37:48.688 INFO:teuthology.orchestra.run.smithi077.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-01-25T05:37:48.688 INFO:teuthology.orchestra.run.smithi077.stdout: "client_die_on_failed_remount": "false", 2023-01-25T05:37:48.689 INFO:teuthology.orchestra.run.smithi077.stdout: "client_dirsize_rbytes": "true", 2023-01-25T05:37:48.689 INFO:teuthology.orchestra.run.smithi077.stdout: "client_force_lazyio": "false", 2023-01-25T05:37:48.689 INFO:teuthology.orchestra.run.smithi077.stdout: "client_fs": "", 2023-01-25T05:37:48.689 INFO:teuthology.orchestra.run.smithi077.stdout: "client_inject_fixed_oldest_tid": "false", 2023-01-25T05:37:48.689 INFO:teuthology.orchestra.run.smithi077.stdout: "client_inject_release_failure": "false", 2023-01-25T05:37:48.690 INFO:teuthology.orchestra.run.smithi077.stdout: "client_max_inline_size": "4096", 2023-01-25T05:37:48.690 INFO:teuthology.orchestra.run.smithi077.stdout: "client_mds_namespace": "", 2023-01-25T05:37:48.690 INFO:teuthology.orchestra.run.smithi077.stdout: "client_metadata": "", 2023-01-25T05:37:48.690 INFO:teuthology.orchestra.run.smithi077.stdout: "client_mount_gid": "-1", 2023-01-25T05:37:48.690 INFO:teuthology.orchestra.run.smithi077.stdout: "client_mount_timeout": "300", 2023-01-25T05:37:48.690 INFO:teuthology.orchestra.run.smithi077.stdout: "client_mount_uid": "-1", 2023-01-25T05:37:48.691 INFO:teuthology.orchestra.run.smithi077.stdout: "client_mountpoint": "/", 2023-01-25T05:37:48.691 INFO:teuthology.orchestra.run.smithi077.stdout: "client_notify_timeout": "10", 2023-01-25T05:37:48.691 INFO:teuthology.orchestra.run.smithi077.stdout: "client_oc": "true", 2023-01-25T05:37:48.691 INFO:teuthology.orchestra.run.smithi077.stdout: "client_oc_max_dirty": "104857600", 2023-01-25T05:37:48.691 INFO:teuthology.orchestra.run.smithi077.stdout: "client_oc_max_dirty_age": "5.000000", 2023-01-25T05:37:48.692 INFO:teuthology.orchestra.run.smithi077.stdout: "client_oc_max_objects": "1000", 2023-01-25T05:37:48.692 INFO:teuthology.orchestra.run.smithi077.stdout: "client_oc_size": "209715200", 2023-01-25T05:37:48.692 INFO:teuthology.orchestra.run.smithi077.stdout: "client_oc_target_dirty": "8388608", 2023-01-25T05:37:48.692 INFO:teuthology.orchestra.run.smithi077.stdout: "client_permissions": "true", 2023-01-25T05:37:48.692 INFO:teuthology.orchestra.run.smithi077.stdout: "client_quota": "true", 2023-01-25T05:37:48.693 INFO:teuthology.orchestra.run.smithi077.stdout: "client_quota_df": "true", 2023-01-25T05:37:48.693 INFO:teuthology.orchestra.run.smithi077.stdout: "client_readahead_max_bytes": "0", 2023-01-25T05:37:48.693 INFO:teuthology.orchestra.run.smithi077.stdout: "client_readahead_max_periods": "4", 2023-01-25T05:37:48.693 INFO:teuthology.orchestra.run.smithi077.stdout: "client_readahead_min": "131072", 2023-01-25T05:37:48.693 INFO:teuthology.orchestra.run.smithi077.stdout: "client_reconnect_stale": "false", 2023-01-25T05:37:48.693 INFO:teuthology.orchestra.run.smithi077.stdout: "client_shutdown_timeout": "30", 2023-01-25T05:37:48.694 INFO:teuthology.orchestra.run.smithi077.stdout: "client_snapdir": ".snap", 2023-01-25T05:37:48.694 INFO:teuthology.orchestra.run.smithi077.stdout: "client_tick_interval": "1", 2023-01-25T05:37:48.694 INFO:teuthology.orchestra.run.smithi077.stdout: "client_trace": "", 2023-01-25T05:37:48.694 INFO:teuthology.orchestra.run.smithi077.stdout: "client_try_dentry_invalidate": "false", 2023-01-25T05:37:48.694 INFO:teuthology.orchestra.run.smithi077.stdout: "client_use_faked_inos": "false", 2023-01-25T05:37:48.695 INFO:teuthology.orchestra.run.smithi077.stdout: "client_use_random_mds": "false", 2023-01-25T05:37:48.695 INFO:teuthology.orchestra.run.smithi077.stdout: "clog_to_graylog": "false", 2023-01-25T05:37:48.695 INFO:teuthology.orchestra.run.smithi077.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-01-25T05:37:48.695 INFO:teuthology.orchestra.run.smithi077.stdout: "clog_to_graylog_port": "12201", 2023-01-25T05:37:48.695 INFO:teuthology.orchestra.run.smithi077.stdout: "clog_to_monitors": "default=true", 2023-01-25T05:37:48.696 INFO:teuthology.orchestra.run.smithi077.stdout: "clog_to_syslog": "false", 2023-01-25T05:37:48.696 INFO:teuthology.orchestra.run.smithi077.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-01-25T05:37:48.696 INFO:teuthology.orchestra.run.smithi077.stdout: "clog_to_syslog_level": "info", 2023-01-25T05:37:48.696 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_addr": "-", 2023-01-25T05:37:48.696 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_network": "", 2023-01-25T05:37:48.697 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_network_interface": "", 2023-01-25T05:37:48.697 INFO:teuthology.orchestra.run.smithi077.stdout: "compressor_zlib_isal": "false", 2023-01-25T05:37:48.697 INFO:teuthology.orchestra.run.smithi077.stdout: "compressor_zlib_level": "5", 2023-01-25T05:37:48.697 INFO:teuthology.orchestra.run.smithi077.stdout: "compressor_zlib_winsize": "-15", 2023-01-25T05:37:48.697 INFO:teuthology.orchestra.run.smithi077.stdout: "compressor_zstd_level": "1", 2023-01-25T05:37:48.697 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-01-25T05:37:48.698 INFO:teuthology.orchestra.run.smithi077.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-01-25T05:37:48.698 INFO:teuthology.orchestra.run.smithi077.stdout: "crimson_alien_op_num_threads": "6", 2023-01-25T05:37:48.698 INFO:teuthology.orchestra.run.smithi077.stdout: "crimson_alien_thread_cpu_cores": "", 2023-01-25T05:37:48.698 INFO:teuthology.orchestra.run.smithi077.stdout: "crimson_osd_obc_lru_size": "10", 2023-01-25T05:37:48.698 INFO:teuthology.orchestra.run.smithi077.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-01-25T05:37:48.699 INFO:teuthology.orchestra.run.smithi077.stdout: "crimson_seastar_smp": "1", 2023-01-25T05:37:48.699 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_location": "", 2023-01-25T05:37:48.699 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_location_hook": "", 2023-01-25T05:37:48.699 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_location_hook_timeout": "10", 2023-01-25T05:37:48.699 INFO:teuthology.orchestra.run.smithi077.stdout: "daemonize": "false", 2023-01-25T05:37:48.700 INFO:teuthology.orchestra.run.smithi077.stdout: "daos_pool": "tank", 2023-01-25T05:37:48.700 INFO:teuthology.orchestra.run.smithi077.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2023-01-25T05:37:48.700 INFO:teuthology.orchestra.run.smithi077.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2023-01-25T05:37:48.700 INFO:teuthology.orchestra.run.smithi077.stdout: "dbstore_db_name_prefix": "dbstore", 2023-01-25T05:37:48.700 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_alienstore": "0/5", 2023-01-25T05:37:48.701 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_allow_any_pool_priority": "false", 2023-01-25T05:37:48.701 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_asok": "1/5", 2023-01-25T05:37:48.701 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_asok_assert_abort": "false", 2023-01-25T05:37:48.701 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_asserts_on_shutdown": "true", 2023-01-25T05:37:48.701 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_auth": "20/20", 2023-01-25T05:37:48.701 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_bdev": "1/3", 2023-01-25T05:37:48.702 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_bluefs": "1/5", 2023-01-25T05:37:48.702 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_bluestore": "1/5", 2023-01-25T05:37:48.702 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_buffer": "0/1", 2023-01-25T05:37:48.702 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_ceph_exporter": "1/5", 2023-01-25T05:37:48.702 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_cephfs_mirror": "0/5", 2023-01-25T05:37:48.703 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_cephsqlite": "0/5", 2023-01-25T05:37:48.703 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_client": "0/5", 2023-01-25T05:37:48.703 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_compressor": "1/5", 2023-01-25T05:37:48.703 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_context": "0/1", 2023-01-25T05:37:48.703 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_crush": "1/1", 2023-01-25T05:37:48.704 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_crypto": "1/5", 2023-01-25T05:37:48.704 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_cyanstore": "0/5", 2023-01-25T05:37:48.704 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_deliberately_leak_memory": "false", 2023-01-25T05:37:48.704 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_disable_randomized_ping": "false", 2023-01-25T05:37:48.704 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_dpdk": "1/5", 2023-01-25T05:37:48.704 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_eventtrace": "1/5", 2023-01-25T05:37:48.705 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_filer": "0/1", 2023-01-25T05:37:48.705 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_filestore": "1/3", 2023-01-25T05:37:48.705 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_finisher": "1/1", 2023-01-25T05:37:48.705 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_fuse": "1/5", 2023-01-25T05:37:48.705 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_heartbeat_testing_span": "0", 2023-01-25T05:37:48.706 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_heartbeatmap": "1/5", 2023-01-25T05:37:48.706 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_immutable_obj_cache": "0/5", 2023-01-25T05:37:48.706 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_javaclient": "1/5", 2023-01-25T05:37:48.706 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_journal": "1/3", 2023-01-25T05:37:48.706 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_journaler": "0/5", 2023-01-25T05:37:48.707 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_kstore": "1/5", 2023-01-25T05:37:48.707 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_leveldb": "4/5", 2023-01-25T05:37:48.707 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_lockdep": "0/1", 2023-01-25T05:37:48.707 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_mclock": "1/5", 2023-01-25T05:37:48.707 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_mds": "1/5", 2023-01-25T05:37:48.707 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_mds_balancer": "1/5", 2023-01-25T05:37:48.708 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_mds_locker": "1/5", 2023-01-25T05:37:48.708 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_mds_log": "1/5", 2023-01-25T05:37:48.708 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_mds_log_expire": "1/5", 2023-01-25T05:37:48.708 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_mds_migrator": "1/5", 2023-01-25T05:37:48.708 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_memstore": "1/5", 2023-01-25T05:37:48.709 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_mgr": "2/5", 2023-01-25T05:37:48.709 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_mgrc": "1/5", 2023-01-25T05:37:48.709 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_mon": "20/20", 2023-01-25T05:37:48.709 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_monc": "0/10", 2023-01-25T05:37:48.709 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_ms": "1/1", 2023-01-25T05:37:48.709 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_none": "0/5", 2023-01-25T05:37:48.710 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_objclass": "0/5", 2023-01-25T05:37:48.710 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_objectcacher": "0/5", 2023-01-25T05:37:48.710 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_objecter": "0/1", 2023-01-25T05:37:48.710 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_optracker": "0/5", 2023-01-25T05:37:48.710 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_osd": "1/5", 2023-01-25T05:37:48.711 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_paxos": "20/20", 2023-01-25T05:37:48.711 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_perfcounter": "1/5", 2023-01-25T05:37:48.711 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_prioritycache": "1/5", 2023-01-25T05:37:48.711 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rados": "0/5", 2023-01-25T05:37:48.711 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rbd": "0/5", 2023-01-25T05:37:48.712 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rbd_mirror": "0/5", 2023-01-25T05:37:48.712 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rbd_pwl": "0/5", 2023-01-25T05:37:48.712 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rbd_replay": "0/5", 2023-01-25T05:37:48.712 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_refs": "0/0", 2023-01-25T05:37:48.712 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_reserver": "1/1", 2023-01-25T05:37:48.713 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rgw": "1/5", 2023-01-25T05:37:48.713 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rgw_access": "1/5", 2023-01-25T05:37:48.713 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rgw_datacache": "1/5", 2023-01-25T05:37:48.713 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rgw_dbstore": "1/5", 2023-01-25T05:37:48.713 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rgw_flight": "1/5", 2023-01-25T05:37:48.714 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rgw_sync": "1/5", 2023-01-25T05:37:48.714 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_rocksdb": "4/5", 2023-01-25T05:37:48.714 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore": "0/5", 2023-01-25T05:37:48.714 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_backref": "0/5", 2023-01-25T05:37:48.714 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_cache": "0/5", 2023-01-25T05:37:48.715 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_cleaner": "0/5", 2023-01-25T05:37:48.715 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_device": "0/5", 2023-01-25T05:37:48.715 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_epm": "0/5", 2023-01-25T05:37:48.715 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_fixedkv_tree": "0/5", 2023-01-25T05:37:48.715 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_journal": "0/5", 2023-01-25T05:37:48.715 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_lba": "0/5", 2023-01-25T05:37:48.716 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_odata": "0/5", 2023-01-25T05:37:48.716 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_omap": "0/5", 2023-01-25T05:37:48.716 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_onode": "0/5", 2023-01-25T05:37:48.716 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_t": "0/5", 2023-01-25T05:37:48.716 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_seastore_tm": "0/5", 2023-01-25T05:37:48.717 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_striper": "0/1", 2023-01-25T05:37:48.717 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_test": "0/5", 2023-01-25T05:37:48.717 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_throttle": "1/1", 2023-01-25T05:37:48.717 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_timer": "0/1", 2023-01-25T05:37:48.717 INFO:teuthology.orchestra.run.smithi077.stdout: "debug_tp": "0/5", 2023-01-25T05:37:48.718 INFO:teuthology.orchestra.run.smithi077.stdout: "device_failure_prediction_mode": "none", 2023-01-25T05:37:48.718 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-01-25T05:37:48.718 INFO:teuthology.orchestra.run.smithi077.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-01-25T05:37:48.718 INFO:teuthology.orchestra.run.smithi077.stdout: "err_to_graylog": "false", 2023-01-25T05:37:48.718 INFO:teuthology.orchestra.run.smithi077.stdout: "err_to_journald": "false", 2023-01-25T05:37:48.719 INFO:teuthology.orchestra.run.smithi077.stdout: "err_to_stderr": "true", 2023-01-25T05:37:48.719 INFO:teuthology.orchestra.run.smithi077.stdout: "err_to_syslog": "false", 2023-01-25T05:37:48.719 INFO:teuthology.orchestra.run.smithi077.stdout: "event_tracing": "false", 2023-01-25T05:37:48.719 INFO:teuthology.orchestra.run.smithi077.stdout: "exporter_addr": "0.0.0.0", 2023-01-25T05:37:48.719 INFO:teuthology.orchestra.run.smithi077.stdout: "exporter_http_port": "9926", 2023-01-25T05:37:48.720 INFO:teuthology.orchestra.run.smithi077.stdout: "exporter_prio_limit": "5", 2023-01-25T05:37:48.720 INFO:teuthology.orchestra.run.smithi077.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-01-25T05:37:48.720 INFO:teuthology.orchestra.run.smithi077.stdout: "exporter_sort_metrics": "true", 2023-01-25T05:37:48.720 INFO:teuthology.orchestra.run.smithi077.stdout: "exporter_stats_period": "5", 2023-01-25T05:37:48.720 INFO:teuthology.orchestra.run.smithi077.stdout: "fake_statfs_for_testing": "0", 2023-01-25T05:37:48.721 INFO:teuthology.orchestra.run.smithi077.stdout: "fatal_signal_handlers": "true", 2023-01-25T05:37:48.721 INFO:teuthology.orchestra.run.smithi077.stdout: "filer_max_purge_ops": "10", 2023-01-25T05:37:48.721 INFO:teuthology.orchestra.run.smithi077.stdout: "filer_max_truncate_ops": "128", 2023-01-25T05:37:48.721 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_apply_finisher_threads": "1", 2023-01-25T05:37:48.721 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_blackhole": "false", 2023-01-25T05:37:48.722 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_btrfs_clone_range": "true", 2023-01-25T05:37:48.722 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_btrfs_snap": "true", 2023-01-25T05:37:48.722 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_caller_concurrency": "10", 2023-01-25T05:37:48.722 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_collect_device_partition_information": "true", 2023-01-25T05:37:48.722 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_commit_timeout": "600.000000", 2023-01-25T05:37:48.723 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_debug_inject_read_err": "false", 2023-01-25T05:37:48.723 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_debug_omap_check": "false", 2023-01-25T05:37:48.723 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_debug_random_read_err": "0.000000", 2023-01-25T05:37:48.723 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_debug_verify_split": "false", 2023-01-25T05:37:48.723 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_dump_file": "", 2023-01-25T05:37:48.724 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-01-25T05:37:48.724 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-01-25T05:37:48.724 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_fadvise": "true", 2023-01-25T05:37:48.724 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_fail_eio": "true", 2023-01-25T05:37:48.724 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_fd_cache_shards": "16", 2023-01-25T05:37:48.725 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_fd_cache_size": "128", 2023-01-25T05:37:48.725 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_fiemap": "false", 2023-01-25T05:37:48.725 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_fiemap_threshold": "4096", 2023-01-25T05:37:48.725 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-01-25T05:37:48.725 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_index_retry_probability": "0.000000", 2023-01-25T05:37:48.725 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_inject_stall": "0", 2023-01-25T05:37:48.726 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_journal_parallel": "false", 2023-01-25T05:37:48.726 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_journal_trailing": "false", 2023-01-25T05:37:48.726 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_journal_writeahead": "false", 2023-01-25T05:37:48.726 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_kill_at": "0", 2023-01-25T05:37:48.726 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-01-25T05:37:48.727 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_inline_xattr_size": "0", 2023-01-25T05:37:48.727 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-01-25T05:37:48.727 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-01-25T05:37:48.727 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-01-25T05:37:48.727 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_inline_xattrs": "0", 2023-01-25T05:37:48.728 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-01-25T05:37:48.728 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_inline_xattrs_other": "2", 2023-01-25T05:37:48.728 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-01-25T05:37:48.728 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_sync_interval": "5.000000", 2023-01-25T05:37:48.728 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_xattr_value_size": "0", 2023-01-25T05:37:48.729 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-01-25T05:37:48.729 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-01-25T05:37:48.729 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-01-25T05:37:48.729 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_merge_threshold": "-10", 2023-01-25T05:37:48.729 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_min_sync_interval": "0.010000", 2023-01-25T05:37:48.730 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_odsync_write": "false", 2023-01-25T05:37:48.730 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_omap_backend": "rocksdb", 2023-01-25T05:37:48.730 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_omap_backend_path": "", 2023-01-25T05:37:48.730 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_omap_header_cache_size": "1024", 2023-01-25T05:37:48.730 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_ondisk_finisher_threads": "1", 2023-01-25T05:37:48.730 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-01-25T05:37:48.731 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_op_thread_timeout": "60", 2023-01-25T05:37:48.731 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_op_threads": "2", 2023-01-25T05:37:48.731 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_punch_hole": "false", 2023-01-25T05:37:48.731 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-01-25T05:37:48.731 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-01-25T05:37:48.732 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-01-25T05:37:48.732 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-01-25T05:37:48.732 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-01-25T05:37:48.732 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_queue_max_bytes": "104857600", 2023-01-25T05:37:48.732 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-01-25T05:37:48.733 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-01-25T05:37:48.733 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-01-25T05:37:48.733 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_queue_max_ops": "50", 2023-01-25T05:37:48.733 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-01-25T05:37:48.733 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_seek_data_hole": "false", 2023-01-25T05:37:48.733 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_sloppy_crc": "false", 2023-01-25T05:37:48.734 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-01-25T05:37:48.734 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_splice": "false", 2023-01-25T05:37:48.734 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_split_multiple": "2", 2023-01-25T05:37:48.734 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_split_rand_factor": "20", 2023-01-25T05:37:48.734 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_update_to": "1000", 2023-01-25T05:37:48.735 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-01-25T05:37:48.735 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-01-25T05:37:48.735 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-01-25T05:37:48.735 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-01-25T05:37:48.735 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-01-25T05:37:48.736 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-01-25T05:37:48.736 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_enable": "true", 2023-01-25T05:37:48.736 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-01-25T05:37:48.736 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-01-25T05:37:48.736 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-01-25T05:37:48.737 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-01-25T05:37:48.737 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-01-25T05:37:48.737 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-01-25T05:37:48.737 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_xfs_extsize": "false", 2023-01-25T05:37:48.737 INFO:teuthology.orchestra.run.smithi077.stdout: "filestore_zfs_snap": "false", 2023-01-25T05:37:48.737 INFO:teuthology.orchestra.run.smithi077.stdout: "fio_dir": "/tmp/fio", 2023-01-25T05:37:48.738 INFO:teuthology.orchestra.run.smithi077.stdout: "fsid": "7bfb8493-ac34-42fb-8797-f07adda855bd", 2023-01-25T05:37:48.738 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_allow_other": "true", 2023-01-25T05:37:48.738 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_atomic_o_trunc": "true", 2023-01-25T05:37:48.738 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_big_writes": "true", 2023-01-25T05:37:48.738 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_debug": "false", 2023-01-25T05:37:48.739 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_default_permissions": "false", 2023-01-25T05:37:48.739 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_disable_pagecache": "false", 2023-01-25T05:37:48.739 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_max_write": "0", 2023-01-25T05:37:48.739 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_multithreaded": "true", 2023-01-25T05:37:48.739 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_require_active_mds": "true", 2023-01-25T05:37:48.740 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_set_user_groups": "true", 2023-01-25T05:37:48.740 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_splice_move": "true", 2023-01-25T05:37:48.740 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_splice_read": "true", 2023-01-25T05:37:48.740 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_splice_write": "true", 2023-01-25T05:37:48.740 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_syncfs_on_mksnap": "true", 2023-01-25T05:37:48.741 INFO:teuthology.orchestra.run.smithi077.stdout: "fuse_use_invalidate_cb": "true", 2023-01-25T05:37:48.741 INFO:teuthology.orchestra.run.smithi077.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-01-25T05:37:48.741 INFO:teuthology.orchestra.run.smithi077.stdout: "gss_target_name": "ceph", 2023-01-25T05:37:48.741 INFO:teuthology.orchestra.run.smithi077.stdout: "heartbeat_file": "", 2023-01-25T05:37:48.741 INFO:teuthology.orchestra.run.smithi077.stdout: "heartbeat_inject_failure": "0", 2023-01-25T05:37:48.742 INFO:teuthology.orchestra.run.smithi077.stdout: "heartbeat_interval": "5", 2023-01-25T05:37:48.742 INFO:teuthology.orchestra.run.smithi077.stdout: "host": "smithi077", 2023-01-25T05:37:48.742 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-01-25T05:37:48.742 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-01-25T05:37:48.742 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_max_size": "1073741824", 2023-01-25T05:37:48.743 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-01-25T05:37:48.743 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-01-25T05:37:48.743 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-01-25T05:37:48.743 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-01-25T05:37:48.743 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-01-25T05:37:48.743 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-01-25T05:37:48.744 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-01-25T05:37:48.744 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-01-25T05:37:48.744 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-01-25T05:37:48.744 INFO:teuthology.orchestra.run.smithi077.stdout: "immutable_object_cache_watermark": "0.900000", 2023-01-25T05:37:48.744 INFO:teuthology.orchestra.run.smithi077.stdout: "inject_early_sigterm": "false", 2023-01-25T05:37:48.745 INFO:teuthology.orchestra.run.smithi077.stdout: "jaeger_tracing_enable": "false", 2023-01-25T05:37:48.745 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_aio": "true", 2023-01-25T05:37:48.745 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_align_min_size": "65536", 2023-01-25T05:37:48.745 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_block_align": "true", 2023-01-25T05:37:48.745 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_block_size": "4096", 2023-01-25T05:37:48.746 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_dio": "true", 2023-01-25T05:37:48.746 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_discard": "false", 2023-01-25T05:37:48.746 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_force_aio": "false", 2023-01-25T05:37:48.746 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_ignore_corruption": "false", 2023-01-25T05:37:48.746 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_max_write_bytes": "10485760", 2023-01-25T05:37:48.747 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_max_write_entries": "100", 2023-01-25T05:37:48.747 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_replay_from": "0", 2023-01-25T05:37:48.747 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_throttle_high_multiple": "0.000000", 2023-01-25T05:37:48.747 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-01-25T05:37:48.747 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-01-25T05:37:48.747 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_throttle_max_multiple": "0.000000", 2023-01-25T05:37:48.748 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_write_header_frequency": "0", 2023-01-25T05:37:48.748 INFO:teuthology.orchestra.run.smithi077.stdout: "journal_zero_on_create": "false", 2023-01-25T05:37:48.748 INFO:teuthology.orchestra.run.smithi077.stdout: "journaler_prefetch_periods": "10", 2023-01-25T05:37:48.748 INFO:teuthology.orchestra.run.smithi077.stdout: "journaler_prezero_periods": "5", 2023-01-25T05:37:48.748 INFO:teuthology.orchestra.run.smithi077.stdout: "journaler_write_head_interval": "15", 2023-01-25T05:37:48.749 INFO:teuthology.orchestra.run.smithi077.stdout: "key": "", 2023-01-25T05:37:48.749 INFO:teuthology.orchestra.run.smithi077.stdout: "keyfile": "", 2023-01-25T05:37:48.749 INFO:teuthology.orchestra.run.smithi077.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-01-25T05:37:48.749 INFO:teuthology.orchestra.run.smithi077.stdout: "kstore_backend": "rocksdb", 2023-01-25T05:37:48.749 INFO:teuthology.orchestra.run.smithi077.stdout: "kstore_default_stripe_size": "65536", 2023-01-25T05:37:48.750 INFO:teuthology.orchestra.run.smithi077.stdout: "kstore_fsck_on_mount": "false", 2023-01-25T05:37:48.750 INFO:teuthology.orchestra.run.smithi077.stdout: "kstore_fsck_on_mount_deep": "true", 2023-01-25T05:37:48.750 INFO:teuthology.orchestra.run.smithi077.stdout: "kstore_max_bytes": "67108864", 2023-01-25T05:37:48.750 INFO:teuthology.orchestra.run.smithi077.stdout: "kstore_max_ops": "512", 2023-01-25T05:37:48.750 INFO:teuthology.orchestra.run.smithi077.stdout: "kstore_nid_prealloc": "1024", 2023-01-25T05:37:48.751 INFO:teuthology.orchestra.run.smithi077.stdout: "kstore_onode_map_size": "1024", 2023-01-25T05:37:48.751 INFO:teuthology.orchestra.run.smithi077.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-01-25T05:37:48.751 INFO:teuthology.orchestra.run.smithi077.stdout: "kstore_sync_submit_transaction": "false", 2023-01-25T05:37:48.751 INFO:teuthology.orchestra.run.smithi077.stdout: "kstore_sync_transaction": "false", 2023-01-25T05:37:48.751 INFO:teuthology.orchestra.run.smithi077.stdout: "leveldb_block_size": "65536", 2023-01-25T05:37:48.752 INFO:teuthology.orchestra.run.smithi077.stdout: "leveldb_bloom_size": "0", 2023-01-25T05:37:48.752 INFO:teuthology.orchestra.run.smithi077.stdout: "leveldb_cache_size": "536870912", 2023-01-25T05:37:48.752 INFO:teuthology.orchestra.run.smithi077.stdout: "leveldb_compact_on_mount": "false", 2023-01-25T05:37:48.752 INFO:teuthology.orchestra.run.smithi077.stdout: "leveldb_compression": "false", 2023-01-25T05:37:48.752 INFO:teuthology.orchestra.run.smithi077.stdout: "leveldb_log": "", 2023-01-25T05:37:48.753 INFO:teuthology.orchestra.run.smithi077.stdout: "leveldb_log_to_ceph_log": "true", 2023-01-25T05:37:48.753 INFO:teuthology.orchestra.run.smithi077.stdout: "leveldb_max_open_files": "0", 2023-01-25T05:37:48.753 INFO:teuthology.orchestra.run.smithi077.stdout: "leveldb_paranoid": "false", 2023-01-25T05:37:48.753 INFO:teuthology.orchestra.run.smithi077.stdout: "leveldb_write_buffer_size": "33554432", 2023-01-25T05:37:48.753 INFO:teuthology.orchestra.run.smithi077.stdout: "librados_thread_count": "2", 2023-01-25T05:37:48.753 INFO:teuthology.orchestra.run.smithi077.stdout: "lockdep": "false", 2023-01-25T05:37:48.754 INFO:teuthology.orchestra.run.smithi077.stdout: "lockdep_force_backtrace": "false", 2023-01-25T05:37:48.754 INFO:teuthology.orchestra.run.smithi077.stdout: "log_coarse_timestamps": "true", 2023-01-25T05:37:48.754 INFO:teuthology.orchestra.run.smithi077.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-01-25T05:37:48.754 INFO:teuthology.orchestra.run.smithi077.stdout: "log_flush_on_exit": "false", 2023-01-25T05:37:48.754 INFO:teuthology.orchestra.run.smithi077.stdout: "log_graylog_host": "127.0.0.1", 2023-01-25T05:37:48.755 INFO:teuthology.orchestra.run.smithi077.stdout: "log_graylog_port": "12201", 2023-01-25T05:37:48.755 INFO:teuthology.orchestra.run.smithi077.stdout: "log_max_new": "1000", 2023-01-25T05:37:48.755 INFO:teuthology.orchestra.run.smithi077.stdout: "log_max_recent": "10000", 2023-01-25T05:37:48.755 INFO:teuthology.orchestra.run.smithi077.stdout: "log_stderr_prefix": "", 2023-01-25T05:37:48.755 INFO:teuthology.orchestra.run.smithi077.stdout: "log_stop_at_utilization": "0.970000", 2023-01-25T05:37:48.756 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": "true", 2023-01-25T05:37:48.756 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_graylog": "false", 2023-01-25T05:37:48.756 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_journald": "false", 2023-01-25T05:37:48.756 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_stderr": "false", 2023-01-25T05:37:48.756 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_syslog": "false", 2023-01-25T05:37:48.757 INFO:teuthology.orchestra.run.smithi077.stdout: "max_rotating_auth_attempts": "10", 2023-01-25T05:37:48.757 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_action_on_write_error": "1", 2023-01-25T05:37:48.757 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_alternate_name_max": "8192", 2023-01-25T05:37:48.757 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_asio_thread_count": "2", 2023-01-25T05:37:48.757 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_export_pin": "true", 2023-01-25T05:37:48.757 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_fragment_dirs": "true", 2023-01-25T05:37:48.758 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-01-25T05:37:48.758 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_fragment_interval": "5", 2023-01-25T05:37:48.758 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_fragment_size_max": "100000", 2023-01-25T05:37:48.758 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_idle_threshold": "0.000000", 2023-01-25T05:37:48.758 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_interval": "10", 2023-01-25T05:37:48.759 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_max": "-1", 2023-01-25T05:37:48.759 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_max_until": "-1", 2023-01-25T05:37:48.759 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_merge_size": "50", 2023-01-25T05:37:48.759 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_midchunk": "0.300000", 2023-01-25T05:37:48.759 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_min_rebalance": "0.100000", 2023-01-25T05:37:48.760 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_min_start": "0.200000", 2023-01-25T05:37:48.760 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_minchunk": "0.001000", 2023-01-25T05:37:48.760 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_mode": "0", 2023-01-25T05:37:48.760 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_need_max": "1.200000", 2023-01-25T05:37:48.760 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_need_min": "0.800000", 2023-01-25T05:37:48.761 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-01-25T05:37:48.761 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_sample_interval": "3.000000", 2023-01-25T05:37:48.761 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_split_bits": "3", 2023-01-25T05:37:48.761 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_split_rd": "25000.000000", 2023-01-25T05:37:48.761 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_split_size": "10000", 2023-01-25T05:37:48.762 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_split_wr": "10000.000000", 2023-01-25T05:37:48.762 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_target_decay": "10.000000", 2023-01-25T05:37:48.762 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-01-25T05:37:48.762 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_beacon_grace": "15.000000", 2023-01-25T05:37:48.762 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_beacon_interval": "4.000000", 2023-01-25T05:37:48.763 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_beacon_mon_down_grace": "60", 2023-01-25T05:37:48.763 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache_memory_limit": "4294967296", 2023-01-25T05:37:48.763 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache_mid": "0.700000", 2023-01-25T05:37:48.763 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache_release_free_interval": "10", 2023-01-25T05:37:48.763 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache_reservation": "0.050000", 2023-01-25T05:37:48.763 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-01-25T05:37:48.764 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache_trim_interval": "1", 2023-01-25T05:37:48.764 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache_trim_threshold": "262144", 2023-01-25T05:37:48.764 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-01-25T05:37:48.764 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-01-25T05:37:48.764 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_client_delegate_inos_pct": "50", 2023-01-25T05:37:48.765 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_client_prealloc_inos": "1000", 2023-01-25T05:37:48.765 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-01-25T05:37:48.765 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_connect_bootstrapping": "false", 2023-01-25T05:37:48.765 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_damage_table_max_entries": "10000", 2023-01-25T05:37:48.765 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-01-25T05:37:48.766 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_debug_auth_pins": "false", 2023-01-25T05:37:48.766 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_debug_frag": "false", 2023-01-25T05:37:48.766 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_debug_scatterstat": "false", 2023-01-25T05:37:48.766 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_debug_subtrees": "false", 2023-01-25T05:37:48.766 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_decay_halflife": "5.000000", 2023-01-25T05:37:48.767 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_default_dir_hash": "2", 2023-01-25T05:37:48.767 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_defer_session_stale": "true", 2023-01-25T05:37:48.767 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_deny_all_reconnect": "false", 2023-01-25T05:37:48.767 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_dir_keys_per_op": "16384", 2023-01-25T05:37:48.767 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_dir_max_commit_size": "10", 2023-01-25T05:37:48.767 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_dir_max_entries": "0", 2023-01-25T05:37:48.768 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_dir_prefetch": "true", 2023-01-25T05:37:48.768 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_dirstat_min_interval": "1.000000", 2023-01-25T05:37:48.768 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_dump_cache_after_rejoin": "false", 2023-01-25T05:37:48.768 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_dump_cache_on_map": "false", 2023-01-25T05:37:48.768 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_dump_cache_threshold_file": "0", 2023-01-25T05:37:48.769 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-01-25T05:37:48.769 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_early_reply": "true", 2023-01-25T05:37:48.769 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_enable_op_tracker": "true", 2023-01-25T05:37:48.769 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_enforce_unique_name": "true", 2023-01-25T05:37:48.769 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_export_ephemeral_distributed": "true", 2023-01-25T05:37:48.770 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-01-25T05:37:48.770 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_export_ephemeral_random": "true", 2023-01-25T05:37:48.770 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-01-25T05:37:48.770 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_extraordinary_events_dump_interval": "0", 2023-01-25T05:37:48.770 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_forward_all_requests_to_auth": "false", 2023-01-25T05:37:48.770 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-01-25T05:37:48.771 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_fscrypt_last_block_max_size": "4096", 2023-01-25T05:37:48.771 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-01-25T05:37:48.771 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_health_cache_threshold": "1.500000", 2023-01-25T05:37:48.771 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_health_summarize_threshold": "10", 2023-01-25T05:37:48.771 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_heartbeat_grace": "15.000000", 2023-01-25T05:37:48.772 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_heartbeat_reset_grace": "1000", 2023-01-25T05:37:48.772 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_inject_health_dummy": "false", 2023-01-25T05:37:48.772 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_inject_migrator_session_race": "false", 2023-01-25T05:37:48.772 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-01-25T05:37:48.772 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_join_fs": "", 2023-01-25T05:37:48.773 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_journal_format": "1", 2023-01-25T05:37:48.773 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_kill_create_at": "0", 2023-01-25T05:37:48.773 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_kill_export_at": "0", 2023-01-25T05:37:48.773 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_kill_import_at": "0", 2023-01-25T05:37:48.773 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_kill_journal_at": "0", 2023-01-25T05:37:48.774 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_kill_journal_expire_at": "0", 2023-01-25T05:37:48.774 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_kill_journal_replay_at": "0", 2023-01-25T05:37:48.774 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_kill_link_at": "0", 2023-01-25T05:37:48.774 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_kill_mdstable_at": "0", 2023-01-25T05:37:48.774 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_kill_openc_at": "0", 2023-01-25T05:37:48.775 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_kill_rename_at": "0", 2023-01-25T05:37:48.775 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log_events_per_segment": "1024", 2023-01-25T05:37:48.775 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log_max_events": "-1", 2023-01-25T05:37:48.775 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log_max_segments": "128", 2023-01-25T05:37:48.775 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log_pause": "false", 2023-01-25T05:37:48.776 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log_segment_size": "0", 2023-01-25T05:37:48.776 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log_skip_corrupt_events": "false", 2023-01-25T05:37:48.776 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log_warn_factor": "2.000000", 2023-01-25T05:37:48.776 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_caps_per_client": "1048576", 2023-01-25T05:37:48.776 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_completed_flushes": "100000", 2023-01-25T05:37:48.777 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_completed_requests": "100000", 2023-01-25T05:37:48.777 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_export_size": "20971520", 2023-01-25T05:37:48.777 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_file_recover": "32", 2023-01-25T05:37:48.777 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_purge_files": "64", 2023-01-25T05:37:48.777 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_purge_ops": "8192", 2023-01-25T05:37:48.777 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-01-25T05:37:48.778 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_retries_on_remount_failure": "5", 2023-01-25T05:37:48.778 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-01-25T05:37:48.778 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_snaps_per_dir": "100", 2023-01-25T05:37:48.778 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_max_xattr_pairs_size": "65536", 2023-01-25T05:37:48.778 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_metrics_update_interval": "2", 2023-01-25T05:37:48.779 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_min_caps_per_client": "100", 2023-01-25T05:37:48.779 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_min_caps_working_set": "10000", 2023-01-25T05:37:48.779 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-01-25T05:37:48.779 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_numa_node": "-1", 2023-01-25T05:37:48.779 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-01-25T05:37:48.780 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_op_complaint_time": "30.000000", 2023-01-25T05:37:48.780 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_op_history_duration": "600", 2023-01-25T05:37:48.780 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_op_history_size": "20", 2023-01-25T05:37:48.780 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_op_log_threshold": "5", 2023-01-25T05:37:48.780 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_ping_grace": "15", 2023-01-25T05:37:48.781 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_ping_interval": "5", 2023-01-25T05:37:48.781 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-01-25T05:37:48.781 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-01-25T05:37:48.781 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_recall_max_caps": "30000", 2023-01-25T05:37:48.781 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-01-25T05:37:48.781 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_recall_max_decay_threshold": "131072", 2023-01-25T05:37:48.782 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-01-25T05:37:48.782 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_recall_warning_threshold": "262144", 2023-01-25T05:37:48.782 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_reconnect_timeout": "45.000000", 2023-01-25T05:37:48.782 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_replay_interval": "1.000000", 2023-01-25T05:37:48.782 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-01-25T05:37:48.783 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-01-25T05:37:48.783 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_root_ino_gid": "0", 2023-01-25T05:37:48.783 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_root_ino_uid": "0", 2023-01-25T05:37:48.783 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-01-25T05:37:48.783 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_session_blocklist_on_evict": "true", 2023-01-25T05:37:48.784 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_session_blocklist_on_timeout": "true", 2023-01-25T05:37:48.784 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-01-25T05:37:48.784 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-01-25T05:37:48.784 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2023-01-25T05:37:48.784 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_session_cap_acquisition_throttle": "500000", 2023-01-25T05:37:48.785 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-01-25T05:37:48.785 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-01-25T05:37:48.785 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_shutdown_check": "0", 2023-01-25T05:37:48.785 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_skip_ino": "0", 2023-01-25T05:37:48.785 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sleep_rank_change": "0.000000", 2023-01-25T05:37:48.786 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_snap_max_uid": "4294967294", 2023-01-25T05:37:48.786 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_snap_min_uid": "0", 2023-01-25T05:37:48.786 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_snap_rstat": "false", 2023-01-25T05:37:48.786 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_standby_replay_damaged": "false", 2023-01-25T05:37:48.786 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_symlink_recovery": "true", 2023-01-25T05:37:48.787 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_task_status_update_interval": "2.000000", 2023-01-25T05:37:48.787 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_thrash_exports": "0", 2023-01-25T05:37:48.787 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_thrash_fragments": "0", 2023-01-25T05:37:48.787 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_tick_interval": "5.000000", 2023-01-25T05:37:48.787 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_valgrind_exit": "false", 2023-01-25T05:37:48.788 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_verify_backtrace": "1", 2023-01-25T05:37:48.788 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_verify_scatter": "false", 2023-01-25T05:37:48.788 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_wipe_ino_prealloc": "false", 2023-01-25T05:37:48.788 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_wipe_sessions": "false", 2023-01-25T05:37:48.788 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool_debug": "false", 2023-01-25T05:37:48.789 INFO:teuthology.orchestra.run.smithi077.stdout: "memstore_debug_omit_block_device_write": "false", 2023-01-25T05:37:48.789 INFO:teuthology.orchestra.run.smithi077.stdout: "memstore_device_bytes": "1073741824", 2023-01-25T05:37:48.789 INFO:teuthology.orchestra.run.smithi077.stdout: "memstore_page_set": "false", 2023-01-25T05:37:48.789 INFO:teuthology.orchestra.run.smithi077.stdout: "memstore_page_size": "65536", 2023-01-25T05:37:48.789 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_client_bytes": "134217728", 2023-01-25T05:37:48.790 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_client_messages": "512", 2023-01-25T05:37:48.790 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-01-25T05:37:48.790 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_connect_retry_interval": "1.000000", 2023-01-25T05:37:48.790 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-01-25T05:37:48.790 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-01-25T05:37:48.790 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_disabled_modules": "", 2023-01-25T05:37:48.791 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-01-25T05:37:48.791 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_max_pg_num_change": "128", 2023-01-25T05:37:48.791 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_mds_bytes": "134217728", 2023-01-25T05:37:48.791 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_mds_messages": "128", 2023-01-25T05:37:48.791 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-01-25T05:37:48.792 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_mon_bytes": "134217728", 2023-01-25T05:37:48.792 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_mon_messages": "128", 2023-01-25T05:37:48.792 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_osd_bytes": "536870912", 2023-01-25T05:37:48.792 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_osd_messages": "8192", 2023-01-25T05:37:48.792 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_pool": "true", 2023-01-25T05:37:48.793 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_service_beacon_grace": "60.000000", 2023-01-25T05:37:48.793 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_standby_modules": "true", 2023-01-25T05:37:48.793 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_stats_period": "5", 2023-01-25T05:37:48.793 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_stats_threshold": "5", 2023-01-25T05:37:48.793 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_tick_period": "2", 2023-01-25T05:37:48.794 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-01-25T05:37:48.794 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_accept_timeout_factor": "2.000000", 2023-01-25T05:37:48.794 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_allow_pool_delete": "true", 2023-01-25T05:37:48.794 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_allow_pool_size_one": "true", 2023-01-25T05:37:48.794 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_auth_validate_all_caps": "true", 2023-01-25T05:37:48.795 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-01-25T05:37:48.795 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-01-25T05:37:48.795 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_client_bytes": "104857600", 2023-01-25T05:37:48.795 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_client_directed_command_retry": "5", 2023-01-25T05:37:48.795 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_client_hunt_interval": "3.000000", 2023-01-25T05:37:48.796 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-01-25T05:37:48.796 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-01-25T05:37:48.796 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-01-25T05:37:48.796 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_client_hunt_parallel": "3", 2023-01-25T05:37:48.796 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_client_log_interval": "1.000000", 2023-01-25T05:37:48.797 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-01-25T05:37:48.797 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_client_ping_interval": "10.000000", 2023-01-25T05:37:48.797 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_client_ping_timeout": "30.000000", 2023-01-25T05:37:48.797 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_clock_drift_allowed": "1.000000", 2023-01-25T05:37:48.797 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-01-25T05:37:48.797 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-01-25T05:37:48.798 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cluster_log_file_level": "debug", 2023-01-25T05:37:48.798 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cluster_log_to_file": "true", 2023-01-25T05:37:48.798 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cluster_log_to_graylog": "false", 2023-01-25T05:37:48.798 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-01-25T05:37:48.798 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-01-25T05:37:48.799 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cluster_log_to_journald": "false", 2023-01-25T05:37:48.799 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cluster_log_to_stderr": "false", 2023-01-25T05:37:48.799 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-01-25T05:37:48.799 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-01-25T05:37:48.799 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-01-25T05:37:48.800 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_compact_on_bootstrap": "false", 2023-01-25T05:37:48.800 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_compact_on_start": "false", 2023-01-25T05:37:48.800 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_compact_on_trim": "true", 2023-01-25T05:37:48.800 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_con_tracker_persist_interval": "10", 2023-01-25T05:37:48.800 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_con_tracker_score_halflife": "43200", 2023-01-25T05:37:48.801 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_config_key_max_entry_size": "65536", 2023-01-25T05:37:48.801 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_cpu_threads": "4", 2023-01-25T05:37:48.801 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_crush_min_required_version": "hammer", 2023-01-25T05:37:48.801 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_daemon_bytes": "419430400", 2023-01-25T05:37:48.801 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-01-25T05:37:48.802 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_data_avail_crit": "5", 2023-01-25T05:37:48.802 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_data_avail_warn": "5", 2023-01-25T05:37:48.802 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_data_size_warn": "16106127360", 2023-01-25T05:37:48.802 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_debug_block_osdmap_trim": "false", 2023-01-25T05:37:48.802 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-01-25T05:37:48.802 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_debug_dump_json": "false", 2023-01-25T05:37:48.803 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-01-25T05:37:48.803 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_debug_dump_transactions": "false", 2023-01-25T05:37:48.803 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_debug_extra_checks": "false", 2023-01-25T05:37:48.803 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-01-25T05:37:48.803 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-01-25T05:37:48.804 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_debug_no_require_quincy": "false", 2023-01-25T05:37:48.804 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_debug_no_require_reef": "false", 2023-01-25T05:37:48.804 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-01-25T05:37:48.804 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_delta_reset_interval": "10.000000", 2023-01-25T05:37:48.804 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_dns_srv_name": "ceph-mon", 2023-01-25T05:37:48.805 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_down_mkfs_grace": "120", 2023-01-25T05:37:48.805 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_election_default_strategy": "3", 2023-01-25T05:37:48.805 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_election_timeout": "5.000000", 2023-01-25T05:37:48.805 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-01-25T05:37:48.805 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_elector_ping_divisor": "2", 2023-01-25T05:37:48.806 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_elector_ping_timeout": "2.000000", 2023-01-25T05:37:48.806 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_enable_op_tracker": "true", 2023-01-25T05:37:48.806 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_fake_pool_delete": "false", 2023-01-25T05:37:48.806 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_force_quorum_join": "false", 2023-01-25T05:37:48.806 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_globalid_prealloc": "10000", 2023-01-25T05:37:48.807 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_health_detail_to_clog": "false", 2023-01-25T05:37:48.807 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_health_log_update_period": "5", 2023-01-25T05:37:48.807 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_health_max_detail": "50", 2023-01-25T05:37:48.807 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_health_to_clog": "true", 2023-01-25T05:37:48.807 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_health_to_clog_interval": "600", 2023-01-25T05:37:48.808 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-01-25T05:37:48.808 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_host": "172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790],172.21.15.121", 2023-01-25T05:37:48.808 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_host_override": "", 2023-01-25T05:37:48.808 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_initial_members": "", 2023-01-25T05:37:48.808 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-01-25T05:37:48.809 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-01-25T05:37:48.809 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-01-25T05:37:48.809 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-01-25T05:37:48.809 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_keyvaluedb": "rocksdb", 2023-01-25T05:37:48.809 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_lease": "5.000000", 2023-01-25T05:37:48.809 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-01-25T05:37:48.810 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-01-25T05:37:48.810 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_log_full_interval": "50", 2023-01-25T05:37:48.810 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_log_max": "10000", 2023-01-25T05:37:48.810 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_log_max_summary": "50", 2023-01-25T05:37:48.810 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_max_log_entries_per_event": "4096", 2023-01-25T05:37:48.811 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_max_log_epochs": "500", 2023-01-25T05:37:48.811 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_max_mdsmap_epochs": "500", 2023-01-25T05:37:48.811 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_max_mgrmap_epochs": "500", 2023-01-25T05:37:48.811 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_max_osd": "10000", 2023-01-25T05:37:48.811 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_max_pg_per_osd": "10000", 2023-01-25T05:37:48.811 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_max_pool_pg_num": "65536", 2023-01-25T05:37:48.812 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-01-25T05:37:48.812 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-01-25T05:37:48.812 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_mds_force_trim_to": "0", 2023-01-25T05:37:48.812 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_mds_skip_sanity": "false", 2023-01-25T05:37:48.812 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_memory_autotune": "true", 2023-01-25T05:37:48.813 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_memory_target": "2147483648", 2023-01-25T05:37:48.813 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_mgr_beacon_grace": "30", 2023-01-25T05:37:48.813 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-01-25T05:37:48.813 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_mgr_digest_period": "5", 2023-01-25T05:37:48.813 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_mgr_inactive_grace": "60", 2023-01-25T05:37:48.814 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_mgr_mkfs_grace": "240", 2023-01-25T05:37:48.814 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-01-25T05:37:48.814 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_min_osdmap_epochs": "50", 2023-01-25T05:37:48.814 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_op_complaint_time": "30", 2023-01-25T05:37:48.814 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_op_history_duration": "600", 2023-01-25T05:37:48.815 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_op_history_size": "20", 2023-01-25T05:37:48.815 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_op_history_slow_op_size": "20", 2023-01-25T05:37:48.815 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_op_history_slow_op_threshold": "10", 2023-01-25T05:37:48.815 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_op_log_threshold": "5", 2023-01-25T05:37:48.815 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-01-25T05:37:48.816 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-01-25T05:37:48.816 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-01-25T05:37:48.816 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_auto_mark_in": "false", 2023-01-25T05:37:48.816 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_auto_mark_new_in": "true", 2023-01-25T05:37:48.816 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-01-25T05:37:48.817 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-01-25T05:37:48.817 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_cache_size": "500", 2023-01-25T05:37:48.817 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_cache_size_min": "134217728", 2023-01-25T05:37:48.817 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_crush_smoke_test": "true", 2023-01-25T05:37:48.817 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_destroyed_out_interval": "600", 2023-01-25T05:37:48.817 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_down_out_interval": "600", 2023-01-25T05:37:48.818 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-01-25T05:37:48.818 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-01-25T05:37:48.818 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_force_trim_to": "0", 2023-01-25T05:37:48.818 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_full_ratio": "0.950000", 2023-01-25T05:37:48.818 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-01-25T05:37:48.819 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_laggy_halflife": "3600", 2023-01-25T05:37:48.819 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_laggy_max_interval": "300", 2023-01-25T05:37:48.819 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_laggy_weight": "0.300000", 2023-01-25T05:37:48.819 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-01-25T05:37:48.819 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_max_creating_pgs": "1024", 2023-01-25T05:37:48.820 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_max_initial_pgs": "1024", 2023-01-25T05:37:48.820 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_min_down_reporters": "2", 2023-01-25T05:37:48.820 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-01-25T05:37:48.820 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-01-25T05:37:48.820 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-01-25T05:37:48.821 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_prime_pg_temp": "true", 2023-01-25T05:37:48.821 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-01-25T05:37:48.821 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-01-25T05:37:48.821 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_report_timeout": "900", 2023-01-25T05:37:48.821 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-01-25T05:37:48.821 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-01-25T05:37:48.822 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_warn_num_repaired": "10", 2023-01-25T05:37:48.822 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osd_warn_op_age": "32.000000", 2023-01-25T05:37:48.822 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-01-25T05:37:48.822 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osdmap_full_prune_interval": "2", 2023-01-25T05:37:48.822 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osdmap_full_prune_min": "15", 2023-01-25T05:37:48.823 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_osdmap_full_prune_txsize": "2", 2023-01-25T05:37:48.823 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-01-25T05:37:48.823 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_pg_stuck_threshold": "60", 2023-01-25T05:37:48.823 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-01-25T05:37:48.823 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_pg_warn_min_objects": "10000", 2023-01-25T05:37:48.824 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_pg_warn_min_per_osd": "0", 2023-01-25T05:37:48.824 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-01-25T05:37:48.824 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_pool_quota_crit_threshold": "0", 2023-01-25T05:37:48.824 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_pool_quota_warn_threshold": "0", 2023-01-25T05:37:48.824 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_probe_timeout": "2.000000", 2023-01-25T05:37:48.825 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_reweight_max_change": "0.050000", 2023-01-25T05:37:48.825 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_reweight_max_osds": "4", 2023-01-25T05:37:48.825 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-01-25T05:37:48.825 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-01-25T05:37:48.825 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-01-25T05:37:48.826 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-01-25T05:37:48.826 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-01-25T05:37:48.826 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_scrub_interval": "300", 2023-01-25T05:37:48.826 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_scrub_max_keys": "100", 2023-01-25T05:37:48.826 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_scrub_timeout": "300", 2023-01-25T05:37:48.827 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_session_timeout": "300", 2023-01-25T05:37:48.827 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_smart_report_timeout": "5", 2023-01-25T05:37:48.827 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_stat_smooth_intervals": "6", 2023-01-25T05:37:48.827 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-01-25T05:37:48.827 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_stretch_pool_min_size": "2", 2023-01-25T05:37:48.827 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_stretch_pool_size": "4", 2023-01-25T05:37:48.828 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-01-25T05:37:48.828 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_subscribe_interval": "86400.000000", 2023-01-25T05:37:48.828 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_sync_debug": "false", 2023-01-25T05:37:48.828 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_sync_max_payload_keys": "2000", 2023-01-25T05:37:48.828 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_sync_max_payload_size": "1048576", 2023-01-25T05:37:48.829 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_sync_provider_kill_at": "0", 2023-01-25T05:37:48.829 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_sync_requester_kill_at": "0", 2023-01-25T05:37:48.829 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_sync_timeout": "60.000000", 2023-01-25T05:37:48.829 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_target_pg_per_osd": "100", 2023-01-25T05:37:48.829 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_tick_interval": "5", 2023-01-25T05:37:48.829 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_timecheck_interval": "300.000000", 2023-01-25T05:37:48.830 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-01-25T05:37:48.830 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_use_min_delay_socket": "false", 2023-01-25T05:37:48.830 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_older_version_delay": "604800", 2023-01-25T05:37:48.830 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-01-25T05:37:48.830 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-01-25T05:37:48.831 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-01-25T05:37:48.831 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_filestore_osds": "false", 2023-01-25T05:37:48.831 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-01-25T05:37:48.831 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-01-25T05:37:48.831 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-01-25T05:37:48.832 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_misplaced": "false", 2023-01-25T05:37:48.832 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-01-25T05:37:48.832 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_older_version": "true", 2023-01-25T05:37:48.832 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-01-25T05:37:48.832 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_pool_no_app": "true", 2023-01-25T05:37:48.833 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-01-25T05:37:48.833 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-01-25T05:37:48.833 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-01-25T05:37:48.833 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-01-25T05:37:48.833 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_on_too_few_osds": "false", 2023-01-25T05:37:48.834 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-01-25T05:37:48.834 INFO:teuthology.orchestra.run.smithi077.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-01-25T05:37:48.834 INFO:teuthology.orchestra.run.smithi077.stdout: "monmap": "", 2023-01-25T05:37:48.834 INFO:teuthology.orchestra.run.smithi077.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2023-01-25T05:37:48.834 INFO:teuthology.orchestra.run.smithi077.stdout: "motr_my_en