2023-01-25T04:49:20.598 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-25T04:49:20.599 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136784 --verbose --name yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136784/orig.config.yaml 2023-01-25T04:49:20.651 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-25T04:49:20.714 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136784 branch: wip-yuri-testing-2023-01-23-0926 description: rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} email: yweinstein@radhat.com first_in_suite: false job_id: '7136784' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri-testing-2023-01-23-0926 ceph: conf: global: mon election default strategy: 1 ms type: async osd debug inject dispatch delay duration: 0.1 osd debug inject dispatch delay probability: 0.1 osd_pool_default_min_size: 2 osd_pool_default_size: 2 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 osd backoff on degraded: true osd backoff on peering: true osd blocked scrub grace period: 3600 osd debug verify cached snaps: true osd debug verify missing on start: true osd inject bad map crc probability: 0.1 osd map cache size: 1 osd max backfills: 6 osd max markdown count: 1000 osd mclock override recovery settings: true osd objectstore: filestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub during recovery: false osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd sloppy crc: true osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(REQUEST_STUCK\) - failed to encode map - but it is still running - objects unfound and apparently lost - osd_map_cache_size - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - must scrub before tier agent can activate sha1: b0d2592529629ab4f86fed5991e556a87dcda563 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: flavor: default sha1: b0d2592529629ab4f86fed5991e556a87dcda563 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-yuri-testing-2023-01-23-0926 sha1: b0d2592529629ab4f86fed5991e556a87dcda563 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: b0d2592529629ab4f86fed5991e556a87dcda563 sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri-testing-2023-01-23-0926 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b0d2592529629ab4f86fed5991e556a87dcda563 targets: smithi135.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi178.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: null - ceph: conf: osd: debug monc: 20 - exec: mon.a: - while ! ceph balancer status ; do sleep 1 ; done - ceph balancer mode crush-compat - ceph balancer on - thrashosds: chance_pgnum_grow: 0.25 chance_pgnum_shrink: 0.25 chance_pgpnum_fix: 0.25 chance_test_map_discontinuity: 2 timeout: 1800 - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache readproxy - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache target_max_objects 250 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 pool_snaps: true pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_22:20:59 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-25T04:49:20.714 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa; will attempt to use it 2023-01-25T04:49:20.715 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa/tasks 2023-01-25T04:49:20.715 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-25T04:49:20.717 INFO:teuthology.task.internal:Checking packages... 2023-01-25T04:49:20.744 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'b0d2592529629ab4f86fed5991e556a87dcda563' 2023-01-25T04:49:20.745 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-25T04:49:20.745 INFO:teuthology.packaging:ref: None 2023-01-25T04:49:20.746 INFO:teuthology.packaging:tag: None 2023-01-25T04:49:20.746 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-01-23-0926 2023-01-25T04:49:20.746 INFO:teuthology.packaging:sha1: b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T04:49:20.747 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-01-23-0926 2023-01-25T04:49:20.894 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1959.gb0d25925 2023-01-25T04:49:20.895 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-25T04:49:20.901 INFO:teuthology.task.internal:no buildpackages task found 2023-01-25T04:49:20.901 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-25T04:49:20.950 INFO:teuthology.task.internal:Saving configuration 2023-01-25T04:49:20.968 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-25T04:49:20.984 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-25T04:49:21.015 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136784', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 04:43:55.365800', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-01-25T04:49:21.044 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136784', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 04:43:55.367063', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-01-25T04:49:21.044 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-25T04:49:21.049 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-25T04:49:21.049 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-25T04:49:21.049 INFO:teuthology.run_tasks:Running task console_log... 2023-01-25T04:49:21.154 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6bb7ad5670>, signals=[15]) 2023-01-25T04:49:21.155 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-25T04:49:21.164 INFO:teuthology.task.internal:Opening connections... 2023-01-25T04:49:21.164 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2023-01-25T04:49:21.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T04:49:21.237 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2023-01-25T04:49:21.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T04:49:21.305 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-25T04:49:21.311 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2023-01-25T04:49:21.331 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2023-01-25T04:49:21.332 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2023-01-25T04:49:21.390 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="Red Hat Enterprise Linux" 2023-01-25T04:49:21.390 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="8.6 (Ootpa)" 2023-01-25T04:49:21.391 INFO:teuthology.orchestra.run.smithi135.stdout:ID="rhel" 2023-01-25T04:49:21.391 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="fedora" 2023-01-25T04:49:21.391 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="8.6" 2023-01-25T04:49:21.391 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el8" 2023-01-25T04:49:21.392 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-25T04:49:21.392 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2023-01-25T04:49:21.392 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-25T04:49:21.392 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://www.redhat.com/" 2023-01-25T04:49:21.392 INFO:teuthology.orchestra.run.smithi135.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-25T04:49:21.393 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T04:49:21.393 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:49:21.393 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T04:49:21.393 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-25T04:49:21.393 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-25T04:49:21.394 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-25T04:49:21.394 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2023-01-25T04:49:21.415 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2023-01-25T04:49:21.435 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2023-01-25T04:49:21.435 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2023-01-25T04:49:21.495 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux" 2023-01-25T04:49:21.495 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.6 (Ootpa)" 2023-01-25T04:49:21.495 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel" 2023-01-25T04:49:21.496 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora" 2023-01-25T04:49:21.496 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.6" 2023-01-25T04:49:21.496 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8" 2023-01-25T04:49:21.496 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-25T04:49:21.497 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2023-01-25T04:49:21.497 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-25T04:49:21.497 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/" 2023-01-25T04:49:21.497 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-25T04:49:21.497 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T04:49:21.498 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:49:21.498 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T04:49:21.498 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-25T04:49:21.498 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-25T04:49:21.499 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-25T04:49:21.499 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-01-25T04:49:21.517 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-25T04:49:21.526 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-25T04:49:21.562 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-25T04:49:21.563 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2023-01-25T04:49:21.566 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2023-01-25T04:49:21.585 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-25T04:49:21.591 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-25T04:49:21.592 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2023-01-25T04:49:21.625 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2023-01-25T04:49:21.647 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-25T04:49:21.697 INFO:teuthology.run_tasks:Running task kernel... 2023-01-25T04:49:21.711 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-25T04:49:21.712 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-25T04:49:21.712 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T04:49:21.713 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T04:49:21.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T04:49:21.732 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2023-01-25T04:49:21.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T04:49:21.734 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2023-01-25T04:49:21.791 INFO:teuthology.orchestra.run.smithi178.stdout:4.18.0-372.9.1.el8.x86_64 2023-01-25T04:49:21.791 INFO:teuthology.task.kernel:Running kernel on smithi178: 4.18.0-372.9.1.el8.x86_64 2023-01-25T04:49:21.792 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum install -y kernel 2023-01-25T04:49:21.793 INFO:teuthology.orchestra.run.smithi135.stdout:4.18.0-372.9.1.el8.x86_64 2023-01-25T04:49:21.793 INFO:teuthology.task.kernel:Running kernel on smithi135: 4.18.0-372.9.1.el8.x86_64 2023-01-25T04:49:21.794 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum install -y kernel 2023-01-25T04:49:22.939 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:49:22.940 INFO:teuthology.orchestra.run.smithi178.stdout:Unable to read consumer identity 2023-01-25T04:49:22.979 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:49:22.979 INFO:teuthology.orchestra.run.smithi135.stdout:Unable to read consumer identity 2023-01-25T04:49:24.308 INFO:teuthology.orchestra.run.smithi178.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-01-25T04:49:24.415 INFO:teuthology.orchestra.run.smithi135.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-01-25T04:49:26.524 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 13 MB 00:02 2023-01-25T04:49:26.558 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 6.0 MB/s | 13 MB 00:02 2023-01-25T04:49:29.533 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 427 kB/s | 24 kB 00:00 2023-01-25T04:49:29.566 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 413 kB/s | 24 kB 00:00 2023-01-25T04:49:30.169 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-01-25T04:49:30.227 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:49:30.227 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-01-25T04:49:30.228 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-01-25T04:49:30.228 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:49:30.278 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:49:30.279 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-01-25T04:49:30.280 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:49:30.345 DEBUG:teuthology.orchestra.run.smithi178:> echo no | sudo yum reinstall kernel || true 2023-01-25T04:49:30.408 DEBUG:teuthology.orchestra.run.smithi135:> echo no | sudo yum reinstall kernel || true 2023-01-25T04:49:30.675 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:49:30.675 INFO:teuthology.orchestra.run.smithi178.stdout:Unable to read consumer identity 2023-01-25T04:49:30.738 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:49:30.738 INFO:teuthology.orchestra.run.smithi135.stdout:Unable to read consumer identity 2023-01-25T04:49:30.928 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:01 ago on Wed 25 Jan 2023 04:49:29 AM UTC. 2023-01-25T04:49:30.987 INFO:teuthology.orchestra.run.smithi178.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T04:49:30.991 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:01 ago on Wed 25 Jan 2023 04:49:29 AM UTC. 2023-01-25T04:49:31.028 INFO:teuthology.orchestra.run.smithi178.stderr:Error: No packages marked for reinstall. 2023-01-25T04:49:31.034 INFO:teuthology.orchestra.run.smithi135.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T04:49:31.074 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum reinstall -y kernel || true 2023-01-25T04:49:31.077 INFO:teuthology.orchestra.run.smithi135.stderr:Error: No packages marked for reinstall. 2023-01-25T04:49:31.123 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum reinstall -y kernel || true 2023-01-25T04:49:31.403 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:49:31.403 INFO:teuthology.orchestra.run.smithi178.stdout:Unable to read consumer identity 2023-01-25T04:49:31.451 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:49:31.452 INFO:teuthology.orchestra.run.smithi135.stdout:Unable to read consumer identity 2023-01-25T04:49:31.655 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Jan 2023 04:49:29 AM UTC. 2023-01-25T04:49:31.696 INFO:teuthology.orchestra.run.smithi178.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T04:49:31.701 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Jan 2023 04:49:29 AM UTC. 2023-01-25T04:49:31.735 INFO:teuthology.orchestra.run.smithi178.stderr:Error: No packages marked for reinstall. 2023-01-25T04:49:31.741 INFO:teuthology.orchestra.run.smithi135.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T04:49:31.769 INFO:teuthology.orchestra.run.smithi135.stderr:Error: No packages marked for reinstall. 2023-01-25T04:49:31.787 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel --last | head -n 1 2023-01-25T04:49:31.821 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel --last | head -n 1 2023-01-25T04:49:31.866 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-01-25T04:49:31.868 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-01-25T04:49:31.868 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-25T04:49:31.869 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-25T04:49:31.869 DEBUG:teuthology.orchestra.run.smithi178:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T04:49:31.895 INFO:teuthology.orchestra.run.smithi178.stdout:ttyS1 2023-01-25T04:49:31.900 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-01-25T04:49:31.902 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-01-25T04:49:31.902 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-25T04:49:31.903 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-25T04:49:31.903 DEBUG:teuthology.orchestra.run.smithi135:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T04:49:31.911 DEBUG:teuthology.parallel:result is None 2023-01-25T04:49:31.944 INFO:teuthology.orchestra.run.smithi135.stdout:ttyS1 2023-01-25T04:49:31.961 DEBUG:teuthology.parallel:result is None 2023-01-25T04:49:31.962 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-25T04:49:31.968 INFO:teuthology.task.internal:Creating test directory... 2023-01-25T04:49:31.969 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T04:49:32.004 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T04:49:32.021 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-25T04:49:32.027 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-25T04:49:32.058 INFO:teuthology.task.internal:Creating archive directory... 2023-01-25T04:49:32.058 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T04:49:32.061 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T04:49:32.084 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-25T04:49:32.090 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-25T04:49:32.090 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T04:49:32.121 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T04:49:32.161 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T04:49:32.167 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T04:49:32.171 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T04:49:32.177 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T04:49:32.179 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-25T04:49:32.186 INFO:teuthology.task.internal:Configuring sudo... 2023-01-25T04:49:32.187 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T04:49:32.214 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T04:49:32.241 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-25T04:49:32.249 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-25T04:49:32.250 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T04:49:32.281 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T04:49:32.298 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T04:49:32.377 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T04:49:32.438 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T04:49:32.517 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T04:49:32.562 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T04:49:32.562 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T04:49:32.629 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T04:49:32.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T04:49:32.731 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T04:49:32.811 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T04:49:32.857 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T04:49:32.857 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T04:49:32.924 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2023-01-25T04:49:32.927 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2023-01-25T04:49:33.005 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T04:49:33.046 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T04:49:33.411 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-25T04:49:33.421 INFO:teuthology.task.internal:Starting timer... 2023-01-25T04:49:33.422 INFO:teuthology.run_tasks:Running task pcp... 2023-01-25T04:49:33.476 INFO:teuthology.run_tasks:Running task selinux... 2023-01-25T04:49:33.508 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-25T04:49:33.509 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2023-01-25T04:49:33.562 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2023-01-25T04:49:33.563 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2023-01-25T04:49:33.617 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2023-01-25T04:49:33.619 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-25T04:49:33.619 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2023-01-25T04:49:33.643 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2023-01-25T04:49:33.643 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2023-01-25T04:49:33.669 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2023-01-25T04:49:33.669 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi135.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2023-01-25T04:49:33.670 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T04:49:33.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T04:49:33.714 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T04:49:33.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T04:49:33.741 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-25T04:49:33.741 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2023-01-25T04:49:33.799 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2023-01-25T04:49:33.866 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-25T04:49:33.876 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-01-25T04:49:33.881 INFO:teuthology.repo_utils:Fetching main from origin 2023-01-25T04:49:34.003 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-25T04:49:34.022 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-25T04:49:34.023 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi135.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-25T04:56:22.300 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi135.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2023-01-25T04:56:22.301 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-01-25T04:56:22.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T04:56:22.412 DEBUG:teuthology.orchestra.run.smithi135:> true 2023-01-25T04:56:22.483 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-01-25T04:56:22.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T04:56:22.559 DEBUG:teuthology.orchestra.run.smithi178:> true 2023-01-25T04:56:22.631 INFO:teuthology.run_tasks:Running task clock... 2023-01-25T04:56:22.641 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-25T04:56:22.642 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T04:56:22.642 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T04:56:22.645 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T04:56:22.645 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T04:56:22.674 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T04:56:22.689 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T04:56:22.705 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T04:56:22.720 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T04:56:22.751 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2023-01-25T04:56:22.762 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2023-01-25T04:56:22.777 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T04:56:22.791 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T04:56:22.798 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2023-01-25T04:56:22.810 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2023-01-25T04:56:22.824 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T04:56:22.839 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T04:56:22.866 INFO:teuthology.orchestra.run.smithi135.stderr:bash: ntpq: command not found 2023-01-25T04:56:22.869 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T04:56:22.870 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2023-01-25T04:56:22.870 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:56:22.870 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:56:22.870 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:56:22.871 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:56:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:bash: ntpq: command not found 2023-01-25T04:56:22.909 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T04:56:22.909 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2023-01-25T04:56:22.909 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:56:22.909 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:56:22.910 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:56:22.910 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:56:22.911 INFO:teuthology.run_tasks:Running task install... 2023-01-25T04:56:22.922 DEBUG:teuthology.task.install:project ceph 2023-01-25T04:56:22.922 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}} 2023-01-25T04:56:22.923 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'} 2023-01-25T04:56:22.923 INFO:teuthology.task.install:Using flavor: default 2023-01-25T04:56:22.929 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-25T04:56:22.929 INFO:teuthology.task.install:extra packages: [] 2023-01-25T04:56:22.930 DEBUG:teuthology.orchestra.run.smithi135:> sudo lsb_release -is 2023-01-25T04:56:22.971 INFO:teuthology.orchestra.run.smithi135.stdout:RedHatEnterprise 2023-01-25T04:56:22.972 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T04:56:22.972 DEBUG:teuthology.orchestra.run.smithi178:> sudo lsb_release -is 2023-01-25T04:56:22.973 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'tag': None, 'wait_for_package': False} 2023-01-25T04:56:22.974 DEBUG:teuthology.orchestra.run.smithi135:> sudo lsb_release -is 2023-01-25T04:56:23.015 INFO:teuthology.orchestra.run.smithi178.stdout:RedHatEnterprise 2023-01-25T04:56:23.016 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T04:56:23.017 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'tag': None, 'wait_for_package': False} 2023-01-25T04:56:23.017 DEBUG:teuthology.orchestra.run.smithi178:> sudo lsb_release -is 2023-01-25T04:56:23.052 INFO:teuthology.orchestra.run.smithi135.stdout:RedHatEnterprise 2023-01-25T04:56:23.053 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T04:56:23.054 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T04:56:23.096 INFO:teuthology.orchestra.run.smithi178.stdout:RedHatEnterprise 2023-01-25T04:56:23.097 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T04:56:23.097 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T04:56:23.212 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/ 2023-01-25T04:56:23.212 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1959.gb0d25925 2023-01-25T04:56:23.244 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/ 2023-01-25T04:56:23.244 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1959.gb0d25925 2023-01-25T04:56:23.547 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T04:56:23.547 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T04:56:23.548 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T04:56:23.577 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T04:56:23.577 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T04:56:23.577 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T04:56:23.579 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T04:56:23.579 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0d2592529629ab4f86fed5991e556a87dcda563/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T04:56:23.606 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T04:56:23.606 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0d2592529629ab4f86fed5991e556a87dcda563/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T04:56:23.647 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T04:56:23.675 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T04:56:23.725 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T04:56:23.752 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T04:56:23.791 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2023-01-25T04:56:23.792 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2023-01-25T04:56:23.818 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2023-01-25T04:56:23.820 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2023-01-25T04:56:24.161 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:56:24.190 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:56:24.436 INFO:teuthology.orchestra.run.smithi135.stdout:44 files removed 2023-01-25T04:56:24.478 INFO:teuthology.orchestra.run.smithi178.stdout:44 files removed 2023-01-25T04:56:24.483 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw 2023-01-25T04:56:24.525 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw 2023-01-25T04:56:24.813 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:56:24.855 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:56:25.978 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 102 kB/s | 80 kB 00:00 2023-01-25T04:56:26.034 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 102 kB/s | 80 kB 00:00 2023-01-25T04:56:26.696 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 14 kB/s | 9.3 kB 00:00 2023-01-25T04:56:26.751 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 14 kB/s | 9.3 kB 00:00 2023-01-25T04:56:27.431 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 2.1 kB/s | 1.5 kB 00:00 2023-01-25T04:56:27.494 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 2.1 kB/s | 1.5 kB 00:00 2023-01-25T04:56:27.572 INFO:teuthology.orchestra.run.smithi135.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-01-25T04:56:27.636 INFO:teuthology.orchestra.run.smithi178.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-01-25T04:56:28.636 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2023-01-25T04:56:29.074 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 13 MB 00:01 2023-01-25T04:56:31.686 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 218 kB/s | 24 kB 00:00 2023-01-25T04:56:32.102 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 315 kB/s | 24 kB 00:00 2023-01-25T04:56:32.596 INFO:teuthology.orchestra.run.smithi178.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-01-25T04:56:33.018 INFO:teuthology.orchestra.run.smithi135.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-01-25T04:56:40.697 INFO:teuthology.orchestra.run.smithi178.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 47 MB 00:00 2023-01-25T04:56:41.036 INFO:teuthology.orchestra.run.smithi135.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-01-25T04:56:51.652 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:56:51.655 INFO:teuthology.orchestra.run.smithi178.stdout:==================================================================================================== 2023-01-25T04:56:51.655 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-01-25T04:56:51.655 INFO:teuthology.orchestra.run.smithi178.stdout:==================================================================================================== 2023-01-25T04:56:51.656 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:56:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 13 M 2023-01-25T04:56:51.656 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2023-01-25T04:56:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T04:56:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T04:56:51.657 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-01-25T04:56:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 5.2 M 2023-01-25T04:56:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 M 2023-01-25T04:56:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 k 2023-01-25T04:56:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T04:56:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 768 k 2023-01-25T04:56:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T04:56:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-25T04:56:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 474 k 2023-01-25T04:56:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-01-25T04:56:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.5 M 2023-01-25T04:56:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T04:56:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-25T04:56:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 45 k 2023-01-25T04:56:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 115 k 2023-01-25T04:56:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 190 k 2023-01-25T04:56:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-25T04:56:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 372 k 2023-01-25T04:56:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 364 k 2023-01-25T04:56:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 107 k 2023-01-25T04:56:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T04:56:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:56:51.661 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:56:51.661 INFO:teuthology.orchestra.run.smithi178.stdout:==================================================================================================== 2023-01-25T04:56:51.661 INFO:teuthology.orchestra.run.smithi178.stdout:Install 21 Packages 2023-01-25T04:56:51.661 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 2 Packages 2023-01-25T04:56:51.662 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:56:51.662 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 62 M 2023-01-25T04:56:51.663 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:56:52.067 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:56:52.069 INFO:teuthology.orchestra.run.smithi135.stdout:==================================================================================================== 2023-01-25T04:56:52.069 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-01-25T04:56:52.070 INFO:teuthology.orchestra.run.smithi135.stdout:==================================================================================================== 2023-01-25T04:56:52.070 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:56:52.070 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 13 M 2023-01-25T04:56:52.070 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2023-01-25T04:56:52.070 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T04:56:52.071 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T04:56:52.071 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-01-25T04:56:52.071 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 5.2 M 2023-01-25T04:56:52.071 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 M 2023-01-25T04:56:52.072 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 k 2023-01-25T04:56:52.072 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T04:56:52.072 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 768 k 2023-01-25T04:56:52.072 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T04:56:52.073 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-25T04:56:52.073 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 474 k 2023-01-25T04:56:52.073 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-01-25T04:56:52.073 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.5 M 2023-01-25T04:56:52.073 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T04:56:52.074 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-25T04:56:52.074 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 45 k 2023-01-25T04:56:52.074 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 115 k 2023-01-25T04:56:52.074 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 190 k 2023-01-25T04:56:52.074 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-25T04:56:52.075 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 372 k 2023-01-25T04:56:52.075 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 364 k 2023-01-25T04:56:52.075 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 107 k 2023-01-25T04:56:52.075 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T04:56:52.075 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:56:52.076 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:56:52.076 INFO:teuthology.orchestra.run.smithi135.stdout:==================================================================================================== 2023-01-25T04:56:52.076 INFO:teuthology.orchestra.run.smithi135.stdout:Install 21 Packages 2023-01-25T04:56:52.076 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 2 Packages 2023-01-25T04:56:52.077 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:56:52.077 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 62 M 2023-01-25T04:56:52.077 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:56:52.329 INFO:teuthology.orchestra.run.smithi178.stdout:(1/23): ceph-base-18.0.0-1959.gb0d25925.el8.x86 7.8 MB/s | 5.2 MB 00:00 2023-01-25T04:56:52.406 INFO:teuthology.orchestra.run.smithi178.stdout:(2/23): ceph-selinux-18.0.0-1959.gb0d25925.el8. 324 kB/s | 24 kB 00:00 2023-01-25T04:56:52.872 INFO:teuthology.orchestra.run.smithi178.stdout:(3/23): libcephfs2-18.0.0-1959.gb0d25925.el8.x8 1.6 MB/s | 768 kB 00:00 2023-01-25T04:56:52.963 INFO:teuthology.orchestra.run.smithi178.stdout:(4/23): libradosstriper1-18.0.0-1959.gb0d25925. 5.1 MB/s | 474 kB 00:00 2023-01-25T04:56:53.872 INFO:teuthology.orchestra.run.smithi178.stdout:(5/23): ceph-radosgw-18.0.0-1959.gb0d25925.el8. 5.9 MB/s | 13 MB 00:02 2023-01-25T04:56:53.956 INFO:teuthology.orchestra.run.smithi178.stdout:(6/23): librgw2-18.0.0-1959.gb0d25925.el8.x86_6 6.5 MB/s | 6.5 MB 00:00 2023-01-25T04:56:53.981 INFO:teuthology.orchestra.run.smithi178.stdout:(7/23): python3-ceph-argparse-18.0.0-1959.gb0d2 411 kB/s | 45 kB 00:00 2023-01-25T04:56:54.006 INFO:teuthology.orchestra.run.smithi178.stdout:(8/23): python3-ceph-common-18.0.0-1959.gb0d259 2.2 MB/s | 115 kB 00:00 2023-01-25T04:56:54.057 INFO:teuthology.orchestra.run.smithi178.stdout:(9/23): python3-cephfs-18.0.0-1959.gb0d25925.el 2.5 MB/s | 190 kB 00:00 2023-01-25T04:56:54.090 INFO:teuthology.orchestra.run.smithi178.stdout:(10/23): python3-rados-18.0.0-1959.gb0d25925.el 4.3 MB/s | 372 kB 00:00 2023-01-25T04:56:54.157 INFO:teuthology.orchestra.run.smithi178.stdout:(11/23): python3-rbd-18.0.0-1959.gb0d25925.el8. 3.6 MB/s | 364 kB 00:00 2023-01-25T04:56:54.182 INFO:teuthology.orchestra.run.smithi178.stdout:(12/23): python3-rgw-18.0.0-1959.gb0d25925.el8. 1.1 MB/s | 107 kB 00:00 2023-01-25T04:56:54.424 INFO:teuthology.orchestra.run.smithi178.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 243 kB/s | 59 kB 00:00 2023-01-25T04:56:54.475 INFO:teuthology.orchestra.run.smithi178.stdout:(14/23): gperftools-libs-2.7-9.el8.x86_64.rpm 965 kB/s | 306 kB 00:00 2023-01-25T04:56:54.509 INFO:teuthology.orchestra.run.smithi178.stdout:(15/23): libunwind-1.3.1-3.el8.x86_64.rpm 894 kB/s | 75 kB 00:00 2023-01-25T04:56:54.559 INFO:teuthology.orchestra.run.smithi178.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 940 kB/s | 47 kB 00:00 2023-01-25T04:56:54.600 INFO:teuthology.orchestra.run.smithi178.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-01-25T04:56:54.634 INFO:teuthology.orchestra.run.smithi178.stdout:(18/23): python3-prettytable-0.7.2-14.el8.noarc 1.3 MB/s | 44 kB 00:00 2023-01-25T04:56:54.676 INFO:teuthology.orchestra.run.smithi178.stdout:(19/23): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-01-25T04:56:54.718 INFO:teuthology.orchestra.run.smithi178.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 7.1 MB/s | 1.7 MB 00:00 2023-01-25T04:56:54.795 INFO:teuthology.orchestra.run.smithi135.stdout:(1/23): ceph-base-18.0.0-1959.gb0d25925.el8.x86 1.9 MB/s | 5.2 MB 00:02 2023-01-25T04:56:54.956 INFO:teuthology.orchestra.run.smithi135.stdout:(2/23): ceph-radosgw-18.0.0-1959.gb0d25925.el8. 4.5 MB/s | 13 MB 00:02 2023-01-25T04:56:54.960 INFO:teuthology.orchestra.run.smithi178.stdout:(21/23): ceph-common-18.0.0-1959.gb0d25925.el8. 7.4 MB/s | 24 MB 00:03 2023-01-25T04:56:54.987 INFO:teuthology.orchestra.run.smithi135.stdout:(3/23): ceph-selinux-18.0.0-1959.gb0d25925.el8. 126 kB/s | 24 kB 00:00 2023-01-25T04:56:55.104 INFO:teuthology.orchestra.run.smithi135.stdout:(4/23): libcephfs2-18.0.0-1959.gb0d25925.el8.x8 5.1 MB/s | 768 kB 00:00 2023-01-25T04:56:55.194 INFO:teuthology.orchestra.run.smithi178.stdout:(22/23): librados2-18.0.0-1959.gb0d25925.el8.x8 7.0 MB/s | 3.6 MB 00:00 2023-01-25T04:56:55.452 INFO:teuthology.orchestra.run.smithi178.stdout:(23/23): librbd1-18.0.0-1959.gb0d25925.el8.x86_ 4.9 MB/s | 3.6 MB 00:00 2023-01-25T04:56:55.454 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:56:55.454 INFO:teuthology.orchestra.run.smithi178.stdout:Total 16 MB/s | 62 MB 00:03 2023-01-25T04:56:55.498 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:56:55.541 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:56:55.542 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:56:55.613 INFO:teuthology.orchestra.run.smithi135.stdout:(5/23): librgw2-18.0.0-1959.gb0d25925.el8.x86_6 13 MB/s | 6.5 MB 00:00 2023-01-25T04:56:55.663 INFO:teuthology.orchestra.run.smithi135.stdout:(6/23): python3-ceph-argparse-18.0.0-1959.gb0d2 891 kB/s | 45 kB 00:00 2023-01-25T04:56:55.714 INFO:teuthology.orchestra.run.smithi135.stdout:(7/23): python3-ceph-common-18.0.0-1959.gb0d259 2.2 MB/s | 115 kB 00:00 2023-01-25T04:56:55.739 INFO:teuthology.orchestra.run.smithi135.stdout:(8/23): libradosstriper1-18.0.0-1959.gb0d25925. 631 kB/s | 474 kB 00:00 2023-01-25T04:56:55.798 INFO:teuthology.orchestra.run.smithi135.stdout:(9/23): python3-rados-18.0.0-1959.gb0d25925.el8 6.3 MB/s | 372 kB 00:00 2023-01-25T04:56:55.815 INFO:teuthology.orchestra.run.smithi135.stdout:(10/23): python3-cephfs-18.0.0-1959.gb0d25925.e 1.8 MB/s | 190 kB 00:00 2023-01-25T04:56:55.865 INFO:teuthology.orchestra.run.smithi135.stdout:(11/23): python3-rgw-18.0.0-1959.gb0d25925.el8. 2.1 MB/s | 107 kB 00:00 2023-01-25T04:56:55.877 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:56:55.881 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:56:55.966 INFO:teuthology.orchestra.run.smithi135.stdout:(12/23): python3-rbd-18.0.0-1959.gb0d25925.el8. 2.1 MB/s | 364 kB 00:00 2023-01-25T04:56:56.158 INFO:teuthology.orchestra.run.smithi135.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 307 kB/s | 59 kB 00:00 2023-01-25T04:56:56.208 INFO:teuthology.orchestra.run.smithi135.stdout:(14/23): gperftools-libs-2.7-9.el8.x86_64.rpm 894 kB/s | 306 kB 00:00 2023-01-25T04:56:56.233 INFO:teuthology.orchestra.run.smithi135.stdout:(15/23): libunwind-1.3.1-3.el8.x86_64.rpm 994 kB/s | 75 kB 00:00 2023-01-25T04:56:56.284 INFO:teuthology.orchestra.run.smithi135.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 939 kB/s | 47 kB 00:00 2023-01-25T04:56:56.326 INFO:teuthology.orchestra.run.smithi135.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-01-25T04:56:56.384 INFO:teuthology.orchestra.run.smithi135.stdout:(18/23): python3-prettytable-0.7.2-14.el8.noarc 754 kB/s | 44 kB 00:00 2023-01-25T04:56:56.435 INFO:teuthology.orchestra.run.smithi135.stdout:(19/23): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-01-25T04:56:56.477 INFO:teuthology.orchestra.run.smithi135.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 6.5 MB/s | 1.7 MB 00:00 2023-01-25T04:56:56.769 INFO:teuthology.orchestra.run.smithi135.stdout:(21/23): librbd1-18.0.0-1959.gb0d25925.el8.x86_ 12 MB/s | 3.6 MB 00:00 2023-01-25T04:56:57.073 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:56:57.285 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T04:56:57.481 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T04:56:57.486 INFO:teuthology.orchestra.run.smithi135.stdout:(22/23): librados2-18.0.0-1959.gb0d25925.el8.x8 3.5 MB/s | 3.6 MB 00:01 2023-01-25T04:56:57.881 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T04:56:58.056 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T04:56:58.151 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T04:56:58.329 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T04:56:58.561 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2023-01-25T04:56:58.739 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T04:56:59.125 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T04:56:59.314 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T04:56:59.328 INFO:teuthology.orchestra.run.smithi135.stdout:(23/23): ceph-common-18.0.0-1959.gb0d25925.el8. 3.4 MB/s | 24 MB 00:07 2023-01-25T04:56:59.331 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:56:59.331 INFO:teuthology.orchestra.run.smithi135.stdout:Total 8.5 MB/s | 62 MB 00:07 2023-01-25T04:56:59.377 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:56:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T04:56:59.422 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:56:59.423 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:56:59.523 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T04:56:59.588 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T04:56:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/25 2023-01-25T04:56:59.761 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:56:59.765 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:56:59.866 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 10/25 2023-01-25T04:57:00.058 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T04:57:00.256 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T04:57:00.414 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T04:57:00.483 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T04:57:00.615 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2023-01-25T04:57:00.802 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2023-01-25T04:57:00.980 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:57:01.256 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T04:57:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T04:57:01.571 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2023-01-25T04:57:01.755 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T04:57:01.824 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T04:57:01.892 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T04:57:02.038 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 18/25 2023-01-25T04:57:02.118 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T04:57:02.204 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T04:57:02.214 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 19/25 2023-01-25T04:57:02.390 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T04:57:02.656 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2023-01-25T04:57:02.860 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T04:57:03.255 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T04:57:03.692 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T04:57:03.791 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T04:57:03.942 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T04:57:04.040 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T04:57:04.224 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/25 2023-01-25T04:57:04.388 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 10/25 2023-01-25T04:57:04.629 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T04:57:04.844 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T04:57:05.033 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T04:57:05.105 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T04:57:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2023-01-25T04:57:05.467 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2023-01-25T04:57:05.934 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T04:57:06.180 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T04:57:06.286 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2023-01-25T04:57:06.483 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T04:57:06.554 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T04:57:06.765 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T04:57:06.791 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 18/25 2023-01-25T04:57:06.992 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 19/25 2023-01-25T04:57:07.032 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T04:57:07.101 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T04:57:07.263 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T04:57:10.755 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T04:57:10.992 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T04:57:11.557 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T04:57:11.889 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T04:57:11.985 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T04:57:12.183 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T04:57:25.794 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T04:57:25.794 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2023-01-25T04:57:25.794 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2023-01-25T04:57:25.794 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /dev 2023-01-25T04:57:25.795 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /run 2023-01-25T04:57:25.795 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2023-01-25T04:57:25.795 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2023-01-25T04:57:25.795 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2023-01-25T04:57:25.795 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2023-01-25T04:57:25.796 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /dev 2023-01-25T04:57:25.796 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:57:27.128 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T04:57:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T04:57:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T04:57:27.361 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T04:57:27.362 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T04:57:27.798 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T04:57:27.798 INFO:teuthology.orchestra.run.smithi178.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-25T04:57:27.799 INFO:teuthology.orchestra.run.smithi178.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-25T04:57:27.799 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:57:27.954 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 1/25 2023-01-25T04:57:27.955 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2/25 2023-01-25T04:57:27.955 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T04:57:27.955 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T04:57:27.956 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 5/25 2023-01-25T04:57:27.956 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 6/25 2023-01-25T04:57:27.956 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T04:57:27.956 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 8/25 2023-01-25T04:57:27.957 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 9/25 2023-01-25T04:57:27.957 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 10/25 2023-01-25T04:57:27.957 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T04:57:27.957 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T04:57:27.957 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 13/25 2023-01-25T04:57:27.958 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-25T04:57:27.958 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-25T04:57:27.958 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-25T04:57:27.958 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-25T04:57:27.958 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-25T04:57:27.959 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-01-25T04:57:27.959 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 20/25 2023-01-25T04:57:27.959 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 21/25 2023-01-25T04:57:27.959 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T04:57:27.959 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T04:57:27.960 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 24/25 2023-01-25T04:57:29.348 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T04:57:29.349 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:49 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:57:31.702 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T04:57:31.702 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2023-01-25T04:57:31.702 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2023-01-25T04:57:31.703 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /dev 2023-01-25T04:57:31.703 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /run 2023-01-25T04:57:31.703 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2023-01-25T04:57:31.703 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2023-01-25T04:57:31.703 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2023-01-25T04:57:31.704 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2023-01-25T04:57:31.704 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /dev 2023-01-25T04:57:31.704 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:57:31.708 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:57:31.896 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:57:31.897 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2023-01-25T04:57:31.897 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.897 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.897 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:57:31.897 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.898 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T04:57:31.898 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.899 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T04:57:31.899 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T04:57:31.899 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.899 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T04:57:31.899 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.900 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T04:57:31.900 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T04:57:31.900 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.900 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.900 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.901 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T04:57:31.901 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.901 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.901 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:31.901 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T04:57:31.902 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:57:31.902 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:57:32.086 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-test 2023-01-25T04:57:32.431 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:57:33.069 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T04:57:33.090 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T04:57:33.090 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T04:57:33.217 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:53 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:57:33.357 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T04:57:33.357 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T04:57:33.861 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T04:57:33.861 INFO:teuthology.orchestra.run.smithi135.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-25T04:57:33.861 INFO:teuthology.orchestra.run.smithi135.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-25T04:57:33.861 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:57:34.017 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 1/25 2023-01-25T04:57:34.017 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2/25 2023-01-25T04:57:34.017 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T04:57:34.018 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T04:57:34.018 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 5/25 2023-01-25T04:57:34.018 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 6/25 2023-01-25T04:57:34.018 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T04:57:34.018 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 8/25 2023-01-25T04:57:34.019 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 9/25 2023-01-25T04:57:34.019 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 10/25 2023-01-25T04:57:34.019 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T04:57:34.019 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T04:57:34.019 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 13/25 2023-01-25T04:57:34.020 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-25T04:57:34.020 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-25T04:57:34.020 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-25T04:57:34.020 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-25T04:57:34.020 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-25T04:57:34.021 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-01-25T04:57:34.021 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 20/25 2023-01-25T04:57:34.021 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 21/25 2023-01-25T04:57:34.021 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T04:57:34.021 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T04:57:34.022 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 24/25 2023-01-25T04:57:34.593 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:57:34.594 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================ 2023-01-25T04:57:34.594 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-01-25T04:57:34.594 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================ 2023-01-25T04:57:34.594 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:57:34.595 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 47 M 2023-01-25T04:57:34.595 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-01-25T04:57:34.595 INFO:teuthology.orchestra.run.smithi178.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-01-25T04:57:34.595 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 137 k 2023-01-25T04:57:34.595 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-25T04:57:34.596 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-01-25T04:57:34.596 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-25T04:57:34.596 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:57:34.596 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:57:34.596 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================ 2023-01-25T04:57:34.597 INFO:teuthology.orchestra.run.smithi178.stdout:Install 6 Packages 2023-01-25T04:57:34.597 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:57:34.597 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 48 M 2023-01-25T04:57:34.598 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 218 M 2023-01-25T04:57:34.598 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:57:34.632 INFO:teuthology.orchestra.run.smithi178.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.9 MB/s | 70 kB 00:00 2023-01-25T04:57:34.690 INFO:teuthology.orchestra.run.smithi178.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-01-25T04:57:34.732 INFO:teuthology.orchestra.run.smithi178.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-01-25T04:57:34.774 INFO:teuthology.orchestra.run.smithi178.stdout:(4/6): jq-1.6-3.el8.x86_64.rpm 4.9 MB/s | 202 kB 00:00 2023-01-25T04:57:35.066 INFO:teuthology.orchestra.run.smithi178.stdout:(5/6): libcephsqlite-18.0.0-1959.gb0d25925.el8. 292 kB/s | 137 kB 00:00 2023-01-25T04:57:35.452 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T04:57:35.453 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:55 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:57:37.853 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:57:38.046 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:57:38.046 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2023-01-25T04:57:38.046 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.047 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.047 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:57:38.047 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.047 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.047 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.048 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.048 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T04:57:38.048 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.048 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T04:57:38.048 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T04:57:38.049 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.049 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T04:57:38.049 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.049 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T04:57:38.049 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T04:57:38.050 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.050 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.050 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.050 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T04:57:38.050 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.051 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.051 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:38.051 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T04:57:38.051 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:57:38.051 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:57:38.217 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-test 2023-01-25T04:57:38.563 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:57:38.958 INFO:teuthology.orchestra.run.smithi178.stdout:(6/6): ceph-test-18.0.0-1959.gb0d25925.el8.x86_ 11 MB/s | 47 MB 00:04 2023-01-25T04:57:38.960 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:57:38.960 INFO:teuthology.orchestra.run.smithi178.stdout:Total 11 MB/s | 48 MB 00:04 2023-01-25T04:57:38.997 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:57:39.024 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:57:39.024 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:57:39.274 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:57:39.278 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:57:39.374 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:59 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:57:39.705 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:57:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T04:57:39.902 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T04:57:40.101 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : jq-1.6-3.el8.x86_64 2/6 2023-01-25T04:57:40.265 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2023-01-25T04:57:40.410 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-25T04:57:40.590 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T04:57:40.826 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:57:40.827 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2023-01-25T04:57:40.827 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-01-25T04:57:40.827 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2023-01-25T04:57:40.828 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:57:40.828 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 47 M 2023-01-25T04:57:40.828 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-01-25T04:57:40.828 INFO:teuthology.orchestra.run.smithi135.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-01-25T04:57:40.829 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 137 k 2023-01-25T04:57:40.829 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-25T04:57:40.829 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-01-25T04:57:40.829 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-25T04:57:40.829 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:57:40.830 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:57:40.830 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2023-01-25T04:57:40.830 INFO:teuthology.orchestra.run.smithi135.stdout:Install 6 Packages 2023-01-25T04:57:40.830 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:57:40.831 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 48 M 2023-01-25T04:57:40.831 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 218 M 2023-01-25T04:57:40.831 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:57:40.872 INFO:teuthology.orchestra.run.smithi135.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-01-25T04:57:40.939 INFO:teuthology.orchestra.run.smithi135.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-01-25T04:57:40.981 INFO:teuthology.orchestra.run.smithi135.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-01-25T04:57:41.023 INFO:teuthology.orchestra.run.smithi135.stdout:(4/6): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-01-25T04:57:41.073 INFO:teuthology.orchestra.run.smithi135.stdout:(5/6): libcephsqlite-18.0.0-1959.gb0d25925.el8. 563 kB/s | 137 kB 00:00 2023-01-25T04:57:43.532 INFO:teuthology.orchestra.run.smithi135.stdout:(6/6): ceph-test-18.0.0-1959.gb0d25925.el8.x86_ 18 MB/s | 47 MB 00:02 2023-01-25T04:57:43.534 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:57:43.534 INFO:teuthology.orchestra.run.smithi135.stdout:Total 18 MB/s | 48 MB 00:02 2023-01-25T04:57:43.571 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:57:43.592 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:57:43.593 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:57:43.850 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:57:43.854 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:57:44.290 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:57:44.428 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T04:57:44.502 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T04:57:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : jq-1.6-3.el8.x86_64 2/6 2023-01-25T04:57:44.970 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2023-01-25T04:57:45.154 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T04:57:45.164 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-25T04:57:45.361 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T04:57:45.436 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T04:57:45.663 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T04:57:45.663 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 1/6 2023-01-25T04:57:45.663 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2/6 2023-01-25T04:57:45.664 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-25T04:57:45.664 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2023-01-25T04:57:45.664 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-25T04:57:46.739 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : jq-1.6-3.el8.x86_64 6/6 2023-01-25T04:57:46.739 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:06 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:57:49.069 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:57:49.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:57:49.249 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:57:49.249 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:49.249 INFO:teuthology.orchestra.run.smithi178.stdout: jq-1.6-3.el8.x86_64 2023-01-25T04:57:49.249 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:49.250 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T04:57:49.250 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T04:57:49.250 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T04:57:49.250 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:57:49.250 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:57:49.383 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph 2023-01-25T04:57:49.719 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:57:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T04:57:50.274 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T04:57:50.494 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T04:57:50.494 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 1/6 2023-01-25T04:57:50.495 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2/6 2023-01-25T04:57:50.495 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-25T04:57:50.495 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2023-01-25T04:57:50.495 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-25T04:57:50.512 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:10 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:57:51.603 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : jq-1.6-3.el8.x86_64 6/6 2023-01-25T04:57:51.603 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:11 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:57:51.895 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:57:51.899 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================================= 2023-01-25T04:57:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-01-25T04:57:51.899 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================================= 2023-01-25T04:57:51.899 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:57:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.4 k 2023-01-25T04:57:51.900 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-01-25T04:57:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 2.3 M 2023-01-25T04:57:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 1.5 M 2023-01-25T04:57:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 236 k 2023-01-25T04:57:51.901 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.9 M 2023-01-25T04:57:51.901 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 17 M 2023-01-25T04:57:51.901 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T04:57:51.901 INFO:teuthology.orchestra.run.smithi178.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T04:57:51.902 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-01-25T04:57:51.902 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T04:57:51.902 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T04:57:51.902 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-01-25T04:57:51.902 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T04:57:51.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T04:57:51.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T04:57:51.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-25T04:57:51.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-25T04:57:51.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T04:57:51.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T04:57:51.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T04:57:51.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-01-25T04:57:51.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-01-25T04:57:51.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-25T04:57:51.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T04:57:51.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T04:57:51.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T04:57:51.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T04:57:51.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T04:57:51.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T04:57:51.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T04:57:51.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-25T04:57:51.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T04:57:51.906 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:57:51.906 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:57:51.907 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================================= 2023-01-25T04:57:51.907 INFO:teuthology.orchestra.run.smithi178.stdout:Install 32 Packages 2023-01-25T04:57:51.907 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:57:51.908 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 28 M 2023-01-25T04:57:51.908 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 110 M 2023-01-25T04:57:51.909 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:57:52.033 INFO:teuthology.orchestra.run.smithi178.stdout:(1/32): ceph-18.0.0-1959.gb0d25925.el8.x86_64.r 50 kB/s | 6.4 kB 00:00 2023-01-25T04:57:52.276 INFO:teuthology.orchestra.run.smithi178.stdout:(2/32): ceph-mds-18.0.0-1959.gb0d25925.el8.x86_ 6.1 MB/s | 2.3 MB 00:00 2023-01-25T04:57:52.318 INFO:teuthology.orchestra.run.smithi178.stdout:(3/32): ceph-mgr-18.0.0-1959.gb0d25925.el8.x86_ 3.7 MB/s | 1.5 MB 00:00 2023-01-25T04:57:52.410 INFO:teuthology.orchestra.run.smithi178.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1959.gb0d2 2.5 MB/s | 236 kB 00:00 2023-01-25T04:57:52.543 INFO:teuthology.orchestra.run.smithi178.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 327 kB/s | 44 kB 00:00 2023-01-25T04:57:52.652 INFO:teuthology.orchestra.run.smithi178.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.7 MB/s | 185 kB 00:00 2023-01-25T04:57:52.752 INFO:teuthology.orchestra.run.smithi178.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.7 MB/s | 173 kB 00:00 2023-01-25T04:57:52.836 INFO:teuthology.orchestra.run.smithi178.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 4.5 MB/s | 384 kB 00:00 2023-01-25T04:57:52.886 INFO:teuthology.orchestra.run.smithi178.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 223 kB/s | 11 kB 00:00 2023-01-25T04:57:52.936 INFO:teuthology.orchestra.run.smithi178.stdout:(10/32): python3-jaraco-functools-2.0-4.el8.noa 351 kB/s | 18 kB 00:00 2023-01-25T04:57:53.011 INFO:teuthology.orchestra.run.smithi178.stdout:(11/32): ceph-mon-18.0.0-1959.gb0d25925.el8.x86 4.0 MB/s | 3.9 MB 00:00 2023-01-25T04:57:53.037 INFO:teuthology.orchestra.run.smithi178.stdout:(12/32): python3-logutils-0.3.5-11.el8.noarch.r 486 kB/s | 49 kB 00:00 2023-01-25T04:57:53.212 INFO:teuthology.orchestra.run.smithi178.stdout:(13/32): ceph-osd-18.0.0-1959.gb0d25925.el8.x86 18 MB/s | 17 MB 00:00 2023-01-25T04:57:53.237 INFO:teuthology.orchestra.run.smithi178.stdout:(14/32): python3-more-itertools-7.2.0-3.el8.noa 260 kB/s | 59 kB 00:00 2023-01-25T04:57:53.271 INFO:teuthology.orchestra.run.smithi178.stdout:(15/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.2 MB/s | 283 kB 00:00 2023-01-25T04:57:53.296 INFO:teuthology.orchestra.run.smithi178.stdout:(16/32): python3-simplegeneric-0.8.1-17.el8.noa 319 kB/s | 19 kB 00:00 2023-01-25T04:57:53.321 INFO:teuthology.orchestra.run.smithi178.stdout:(17/32): python3-singledispatch-3.4.0.3-18.el8. 478 kB/s | 24 kB 00:00 2023-01-25T04:57:53.347 INFO:teuthology.orchestra.run.smithi178.stdout:(18/32): python3-portend-2.6-1.el8.noarch.rpm 116 kB/s | 16 kB 00:00 2023-01-25T04:57:53.372 INFO:teuthology.orchestra.run.smithi178.stdout:(19/32): python3-tempora-1.14.1-5.el8.noarch.rp 387 kB/s | 29 kB 00:00 2023-01-25T04:57:53.397 INFO:teuthology.orchestra.run.smithi178.stdout:(20/32): python3-trustme-0.6.0-4.el8.noarch.rpm 364 kB/s | 27 kB 00:00 2023-01-25T04:57:53.448 INFO:teuthology.orchestra.run.smithi178.stdout:(21/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.7 MB/s | 85 kB 00:00 2023-01-25T04:57:53.481 INFO:teuthology.orchestra.run.smithi178.stdout:(22/32): python3-waitress-1.2.1-2.el8.1.noarch. 1.8 MB/s | 241 kB 00:00 2023-01-25T04:57:53.506 INFO:teuthology.orchestra.run.smithi178.stdout:(23/32): python3-zc-lockfile-2.0-2.el8.noarch.r 392 kB/s | 23 kB 00:00 2023-01-25T04:57:53.531 INFO:teuthology.orchestra.run.smithi178.stdout:(24/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.5 MB/s | 251 kB 00:00 2023-01-25T04:57:53.582 INFO:teuthology.orchestra.run.smithi178.stdout:(25/32): python3-pycparser-2.14-14.el8.noarch.r 1.4 MB/s | 109 kB 00:00 2023-01-25T04:57:53.607 INFO:teuthology.orchestra.run.smithi178.stdout:(26/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.8 MB/s | 238 kB 00:00 2023-01-25T04:57:53.632 INFO:teuthology.orchestra.run.smithi178.stdout:(27/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-01-25T04:57:53.658 INFO:teuthology.orchestra.run.smithi178.stdout:(28/32): python3-markupsafe-0.23-19.el8.x86_64. 774 kB/s | 39 kB 00:00 2023-01-25T04:57:53.683 INFO:teuthology.orchestra.run.smithi178.stdout:(29/32): python3-mako-1.0.6-13.el8.noarch.rpm 3.0 MB/s | 157 kB 00:00 2023-01-25T04:57:53.716 INFO:teuthology.orchestra.run.smithi178.stdout:(30/32): python3-cryptography-3.2.1-5.el8.x86_6 3.1 MB/s | 559 kB 00:00 2023-01-25T04:57:53.742 INFO:teuthology.orchestra.run.smithi178.stdout:(31/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 1.7 MB/s | 103 kB 00:00 2023-01-25T04:57:53.776 INFO:teuthology.orchestra.run.smithi178.stdout:(32/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.8 MB/s | 457 kB 00:00 2023-01-25T04:57:53.777 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:57:53.777 INFO:teuthology.orchestra.run.smithi178.stdout:Total 15 MB/s | 28 MB 00:01 2023-01-25T04:57:53.868 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:57:53.906 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:57:53.906 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:57:54.027 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:57:54.111 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:57:54.114 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:57:54.209 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:57:54.210 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:57:54.210 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:54.210 INFO:teuthology.orchestra.run.smithi135.stdout: jq-1.6-3.el8.x86_64 2023-01-25T04:57:54.210 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:57:54.211 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T04:57:54.211 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T04:57:54.211 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T04:57:54.211 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:57:54.211 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:57:54.357 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph 2023-01-25T04:57:54.689 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:57:55.203 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:57:55.400 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T04:57:55.489 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:15 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:57:55.686 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T04:57:55.890 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 3/32 2023-01-25T04:57:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/32 2023-01-25T04:57:56.353 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/32 2023-01-25T04:57:56.532 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 6/32 2023-01-25T04:57:56.772 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 7/32 2023-01-25T04:57:56.940 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:57:56.944 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================================= 2023-01-25T04:57:56.944 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-01-25T04:57:56.944 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================================= 2023-01-25T04:57:56.944 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:57:56.945 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.4 k 2023-01-25T04:57:56.945 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-01-25T04:57:56.945 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 2.3 M 2023-01-25T04:57:56.945 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 1.5 M 2023-01-25T04:57:56.946 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 236 k 2023-01-25T04:57:56.946 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.9 M 2023-01-25T04:57:56.946 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 17 M 2023-01-25T04:57:56.946 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T04:57:56.946 INFO:teuthology.orchestra.run.smithi135.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T04:57:56.947 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-01-25T04:57:56.947 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T04:57:56.947 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T04:57:56.947 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-01-25T04:57:56.948 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T04:57:56.948 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T04:57:56.948 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T04:57:56.948 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-25T04:57:56.948 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-25T04:57:56.949 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T04:57:56.949 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T04:57:56.949 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T04:57:56.949 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-01-25T04:57:56.949 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-01-25T04:57:56.950 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-25T04:57:56.950 INFO:teuthology.orchestra.run.smithi135.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T04:57:56.950 INFO:teuthology.orchestra.run.smithi135.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T04:57:56.950 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T04:57:56.950 INFO:teuthology.orchestra.run.smithi135.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T04:57:56.951 INFO:teuthology.orchestra.run.smithi135.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T04:57:56.951 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T04:57:56.951 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T04:57:56.951 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-25T04:57:56.951 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T04:57:56.952 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:57:56.952 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:57:56.952 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================================= 2023-01-25T04:57:56.952 INFO:teuthology.orchestra.run.smithi135.stdout:Install 32 Packages 2023-01-25T04:57:56.952 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:57:56.953 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 28 M 2023-01-25T04:57:56.954 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 110 M 2023-01-25T04:57:56.954 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:57:57.098 INFO:teuthology.orchestra.run.smithi135.stdout:(1/32): ceph-18.0.0-1959.gb0d25925.el8.x86_64.r 43 kB/s | 6.4 kB 00:00 2023-01-25T04:57:57.132 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 8/32 2023-01-25T04:57:57.331 INFO:teuthology.orchestra.run.smithi135.stdout:(2/32): ceph-mds-18.0.0-1959.gb0d25925.el8.x86_ 6.0 MB/s | 2.3 MB 00:00 2023-01-25T04:57:57.365 INFO:teuthology.orchestra.run.smithi135.stdout:(3/32): ceph-mgr-18.0.0-1959.gb0d25925.el8.x86_ 3.7 MB/s | 1.5 MB 00:00 2023-01-25T04:57:57.381 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 9/32 2023-01-25T04:57:57.483 INFO:teuthology.orchestra.run.smithi135.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1959.gb0d2 2.0 MB/s | 236 kB 00:00 2023-01-25T04:57:57.536 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 10/32 2023-01-25T04:57:57.616 INFO:teuthology.orchestra.run.smithi135.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 327 kB/s | 44 kB 00:00 2023-01-25T04:57:57.725 INFO:teuthology.orchestra.run.smithi135.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.7 MB/s | 185 kB 00:00 2023-01-25T04:57:57.738 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 11/32 2023-01-25T04:57:57.775 INFO:teuthology.orchestra.run.smithi135.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 3.4 MB/s | 173 kB 00:00 2023-01-25T04:57:57.859 INFO:teuthology.orchestra.run.smithi135.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 4.5 MB/s | 384 kB 00:00 2023-01-25T04:57:57.909 INFO:teuthology.orchestra.run.smithi135.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 223 kB/s | 11 kB 00:00 2023-01-25T04:57:57.959 INFO:teuthology.orchestra.run.smithi135.stdout:(10/32): python3-jaraco-functools-2.0-4.el8.noa 352 kB/s | 18 kB 00:00 2023-01-25T04:57:57.961 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 12/32 2023-01-25T04:57:58.018 INFO:teuthology.orchestra.run.smithi135.stdout:(11/32): ceph-mon-18.0.0-1959.gb0d25925.el8.x86 4.2 MB/s | 3.9 MB 00:00 2023-01-25T04:57:58.044 INFO:teuthology.orchestra.run.smithi135.stdout:(12/32): python3-logutils-0.3.5-11.el8.noarch.r 581 kB/s | 49 kB 00:00 2023-01-25T04:57:58.182 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2023-01-25T04:57:58.236 INFO:teuthology.orchestra.run.smithi135.stdout:(13/32): ceph-osd-18.0.0-1959.gb0d25925.el8.x86 19 MB/s | 17 MB 00:00 2023-01-25T04:57:58.286 INFO:teuthology.orchestra.run.smithi135.stdout:(14/32): python3-more-itertools-7.2.0-3.el8.noa 220 kB/s | 59 kB 00:00 2023-01-25T04:57:58.311 INFO:teuthology.orchestra.run.smithi135.stdout:(15/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.0 MB/s | 283 kB 00:00 2023-01-25T04:57:58.337 INFO:teuthology.orchestra.run.smithi135.stdout:(16/32): python3-simplegeneric-0.8.1-17.el8.noa 371 kB/s | 19 kB 00:00 2023-01-25T04:57:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 14/32 2023-01-25T04:57:58.362 INFO:teuthology.orchestra.run.smithi135.stdout:(17/32): python3-singledispatch-3.4.0.3-18.el8. 474 kB/s | 24 kB 00:00 2023-01-25T04:57:58.387 INFO:teuthology.orchestra.run.smithi135.stdout:(18/32): python3-portend-2.6-1.el8.noarch.rpm 103 kB/s | 16 kB 00:00 2023-01-25T04:57:58.413 INFO:teuthology.orchestra.run.smithi135.stdout:(19/32): python3-tempora-1.14.1-5.el8.noarch.rp 386 kB/s | 29 kB 00:00 2023-01-25T04:57:58.438 INFO:teuthology.orchestra.run.smithi135.stdout:(20/32): python3-trustme-0.6.0-4.el8.noarch.rpm 363 kB/s | 27 kB 00:00 2023-01-25T04:57:58.489 INFO:teuthology.orchestra.run.smithi135.stdout:(21/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.7 MB/s | 85 kB 00:00 2023-01-25T04:57:58.514 INFO:teuthology.orchestra.run.smithi135.stdout:(22/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.4 MB/s | 251 kB 00:00 2023-01-25T04:57:58.539 INFO:teuthology.orchestra.run.smithi135.stdout:(23/32): python3-waitress-1.2.1-2.el8.1.noarch. 1.6 MB/s | 241 kB 00:00 2023-01-25T04:57:58.564 INFO:teuthology.orchestra.run.smithi135.stdout:(24/32): python3-zc-lockfile-2.0-2.el8.noarch.r 304 kB/s | 23 kB 00:00 2023-01-25T04:57:58.565 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2023-01-25T04:57:58.623 INFO:teuthology.orchestra.run.smithi135.stdout:(25/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2023-01-25T04:57:58.649 INFO:teuthology.orchestra.run.smithi135.stdout:(26/32): python3-pycparser-2.14-14.el8.noarch.r 1.4 MB/s | 109 kB 00:00 2023-01-25T04:57:58.674 INFO:teuthology.orchestra.run.smithi135.stdout:(27/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.0 MB/s | 54 kB 00:00 2023-01-25T04:57:58.699 INFO:teuthology.orchestra.run.smithi135.stdout:(28/32): python3-markupsafe-0.23-19.el8.x86_64. 767 kB/s | 39 kB 00:00 2023-01-25T04:57:58.725 INFO:teuthology.orchestra.run.smithi135.stdout:(29/32): python3-mako-1.0.6-13.el8.noarch.rpm 3.0 MB/s | 157 kB 00:00 2023-01-25T04:57:58.729 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2023-01-25T04:57:58.783 INFO:teuthology.orchestra.run.smithi135.stdout:(30/32): python3-cryptography-3.2.1-5.el8.x86_6 2.6 MB/s | 559 kB 00:00 2023-01-25T04:57:58.809 INFO:teuthology.orchestra.run.smithi135.stdout:(31/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 1.2 MB/s | 103 kB 00:00 2023-01-25T04:57:58.834 INFO:teuthology.orchestra.run.smithi135.stdout:(32/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.3 MB/s | 457 kB 00:00 2023-01-25T04:57:58.836 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:57:58.836 INFO:teuthology.orchestra.run.smithi135.stdout:Total 15 MB/s | 28 MB 00:01 2023-01-25T04:57:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2023-01-25T04:57:58.924 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:57:58.960 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:57:58.961 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:57:59.017 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2023-01-25T04:57:59.163 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:57:59.167 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:57:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2023-01-25T04:57:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2023-01-25T04:57:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T04:57:59.861 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T04:58:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T04:58:00.288 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:58:00.318 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2023-01-25T04:58:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T04:58:00.687 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2023-01-25T04:58:00.833 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T04:58:01.045 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 3/32 2023-01-25T04:58:01.311 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/32 2023-01-25T04:58:01.528 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/32 2023-01-25T04:58:01.703 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T04:58:01.763 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 6/32 2023-01-25T04:58:02.003 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 7/32 2023-01-25T04:58:02.101 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 27/32 2023-01-25T04:58:02.247 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T04:58:02.431 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 8/32 2023-01-25T04:58:02.753 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 9/32 2023-01-25T04:58:02.933 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 10/32 2023-01-25T04:58:03.172 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 11/32 2023-01-25T04:58:03.376 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 12/32 2023-01-25T04:58:03.580 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2023-01-25T04:58:03.742 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T04:58:03.756 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 14/32 2023-01-25T04:58:03.920 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2023-01-25T04:58:03.930 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T04:58:04.117 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2023-01-25T04:58:04.294 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2023-01-25T04:58:04.322 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T04:58:04.455 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2023-01-25T04:58:04.477 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T04:58:04.691 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T04:58:04.750 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2023-01-25T04:58:04.877 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T04:58:04.895 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T04:58:04.992 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T04:58:04.998 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2023-01-25T04:58:05.205 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T04:58:05.355 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T04:58:05.356 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 1/32 2023-01-25T04:58:05.356 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2/32 2023-01-25T04:58:05.356 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 3/32 2023-01-25T04:58:05.356 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 4/32 2023-01-25T04:58:05.357 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 5/32 2023-01-25T04:58:05.357 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 6/32 2023-01-25T04:58:05.357 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2023-01-25T04:58:05.357 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2023-01-25T04:58:05.357 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2023-01-25T04:58:05.358 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2023-01-25T04:58:05.358 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2023-01-25T04:58:05.358 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2023-01-25T04:58:05.358 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2023-01-25T04:58:05.358 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2023-01-25T04:58:05.359 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2023-01-25T04:58:05.359 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2023-01-25T04:58:05.359 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2023-01-25T04:58:05.359 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2023-01-25T04:58:05.359 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2023-01-25T04:58:05.360 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2023-01-25T04:58:05.360 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/32 2023-01-25T04:58:05.360 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2023-01-25T04:58:05.360 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2023-01-25T04:58:05.361 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2023-01-25T04:58:05.361 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 25/32 2023-01-25T04:58:05.361 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 26/32 2023-01-25T04:58:05.361 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 27/32 2023-01-25T04:58:05.361 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/32 2023-01-25T04:58:05.362 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 29/32 2023-01-25T04:58:05.362 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 30/32 2023-01-25T04:58:05.362 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 31/32 2023-01-25T04:58:05.487 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T04:58:05.747 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T04:58:05.959 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2023-01-25T04:58:06.284 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2023-01-25T04:58:06.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 32/32 2023-01-25T04:58:06.723 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:26 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:07.185 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T04:58:07.494 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 27/32 2023-01-25T04:58:07.647 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T04:58:09.055 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:58:09.140 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T04:58:09.239 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:09.240 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:58:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:58:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T04:58:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T04:58:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T04:58:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T04:58:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T04:58:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T04:58:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T04:58:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T04:58:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T04:58:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T04:58:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T04:58:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T04:58:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T04:58:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T04:58:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T04:58:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T04:58:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T04:58:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T04:58:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T04:58:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T04:58:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T04:58:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T04:58:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T04:58:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T04:58:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T04:58:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T04:58:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:09.247 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:58:09.364 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T04:58:09.388 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-base 2023-01-25T04:58:09.734 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:09.748 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T04:58:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T04:58:10.146 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T04:58:10.341 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T04:58:10.359 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T04:58:10.472 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T04:58:10.523 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:30 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:10.823 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T04:58:10.823 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 1/32 2023-01-25T04:58:10.824 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2/32 2023-01-25T04:58:10.824 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 3/32 2023-01-25T04:58:10.824 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 4/32 2023-01-25T04:58:10.824 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 5/32 2023-01-25T04:58:10.824 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 6/32 2023-01-25T04:58:10.825 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2023-01-25T04:58:10.825 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2023-01-25T04:58:10.825 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2023-01-25T04:58:10.825 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2023-01-25T04:58:10.825 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2023-01-25T04:58:10.826 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2023-01-25T04:58:10.826 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2023-01-25T04:58:10.826 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2023-01-25T04:58:10.826 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2023-01-25T04:58:10.827 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2023-01-25T04:58:10.827 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2023-01-25T04:58:10.827 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2023-01-25T04:58:10.827 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2023-01-25T04:58:10.827 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2023-01-25T04:58:10.828 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/32 2023-01-25T04:58:10.828 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2023-01-25T04:58:10.828 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2023-01-25T04:58:10.828 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2023-01-25T04:58:10.828 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 25/32 2023-01-25T04:58:10.829 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 26/32 2023-01-25T04:58:10.830 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 27/32 2023-01-25T04:58:10.830 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/32 2023-01-25T04:58:10.830 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 29/32 2023-01-25T04:58:10.830 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 30/32 2023-01-25T04:58:10.830 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 31/32 2023-01-25T04:58:11.876 INFO:teuthology.orchestra.run.smithi178.stdout:Package ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:58:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:58:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-01-25T04:58:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:58:12.090 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install cephadm 2023-01-25T04:58:12.333 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 32/32 2023-01-25T04:58:12.334 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:32 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:12.433 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:13.218 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:33 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:14.597 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:58:14.597 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:58:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-01-25T04:58:14.598 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:58:14.598 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:58:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 82 k 2023-01-25T04:58:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:14.599 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:58:14.599 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:58:14.599 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2023-01-25T04:58:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:14.599 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 82 k 2023-01-25T04:58:14.600 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 374 k 2023-01-25T04:58:14.600 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:58:14.780 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:58:14.964 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:14.965 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:58:14.965 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:14.965 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:14.965 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:14.966 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:58:14.966 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:14.966 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:14.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T04:58:14.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T04:58:14.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T04:58:14.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T04:58:14.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T04:58:14.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T04:58:14.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T04:58:14.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T04:58:14.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T04:58:14.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T04:58:14.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T04:58:14.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T04:58:14.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T04:58:14.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T04:58:14.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T04:58:14.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T04:58:14.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T04:58:14.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T04:58:14.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T04:58:14.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T04:58:14.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T04:58:14.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T04:58:14.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T04:58:14.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T04:58:14.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T04:58:14.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T04:58:14.971 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:14.972 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:58:15.107 INFO:teuthology.orchestra.run.smithi178.stdout:cephadm-18.0.0-1959.gb0d25925.el8.noarch.rpm 161 kB/s | 82 kB 00:00 2023-01-25T04:58:15.108 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:58:15.108 INFO:teuthology.orchestra.run.smithi178.stdout:Total 161 kB/s | 82 kB 00:00 2023-01-25T04:58:15.109 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:58:15.116 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:58:15.117 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:58:15.119 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:58:15.122 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:58:15.136 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-base 2023-01-25T04:58:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:58:15.482 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:16.302 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:36 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:16.700 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:58:16.848 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:58:17.014 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:58:17.567 INFO:teuthology.orchestra.run.smithi135.stdout:Package ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:58:17.679 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:58:17.680 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-01-25T04:58:17.681 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:58:18.077 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:58:18.078 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:38 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:18.188 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install cephadm 2023-01-25T04:58:18.528 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:19.335 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:39 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:20.694 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:58:20.717 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:58:20.718 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:58:20.718 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-01-25T04:58:20.718 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:58:20.718 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:58:20.719 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 82 k 2023-01-25T04:58:20.719 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:20.719 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:58:20.719 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:58:20.719 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-01-25T04:58:20.720 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:20.720 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 82 k 2023-01-25T04:58:20.720 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 374 k 2023-01-25T04:58:20.720 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:58:20.878 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:20.878 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:58:20.878 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:58:20.878 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:20.879 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:58:21.012 INFO:teuthology.orchestra.run.smithi135.stdout:cephadm-18.0.0-1959.gb0d25925.el8.noarch.rpm 280 kB/s | 82 kB 00:00 2023-01-25T04:58:21.013 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:58:21.013 INFO:teuthology.orchestra.run.smithi135.stdout:Total 279 kB/s | 82 kB 00:00 2023-01-25T04:58:21.014 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:58:21.021 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:58:21.021 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:58:21.023 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:58:21.027 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T04:58:21.028 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:58:21.072 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:58:21.383 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:22.172 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:42 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:22.680 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:58:22.874 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:58:23.039 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:58:23.569 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:58:23.570 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:58:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2023-01-25T04:58:23.570 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:58:23.570 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:58:23.571 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 166 k 2023-01-25T04:58:23.571 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:23.571 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:58:23.571 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:58:23.572 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2023-01-25T04:58:23.572 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:23.572 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 166 k 2023-01-25T04:58:23.572 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 503 k 2023-01-25T04:58:23.572 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:58:23.770 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-immutable-object-cache-18.0.0-1959.gb0d259 829 kB/s | 166 kB 00:00 2023-01-25T04:58:23.771 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:58:23.772 INFO:teuthology.orchestra.run.smithi178.stdout:Total 824 kB/s | 166 kB 00:00 2023-01-25T04:58:23.772 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:58:23.783 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:58:23.783 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:58:23.835 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:58:23.839 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:58:23.955 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:58:24.099 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T04:58:24.163 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:58:24.163 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:44 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T04:58:25.502 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T04:58:25.503 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:45 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:26.768 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:58:26.948 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:26.949 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:58:26.949 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:58:26.949 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:26.949 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:58:27.095 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T04:58:27.450 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:28.279 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:48 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:28.310 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:58:28.489 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:28.489 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:58:28.490 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:28.490 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:28.490 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:58:28.619 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr 2023-01-25T04:58:28.964 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:29.658 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:58:29.659 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:58:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2023-01-25T04:58:29.659 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:58:29.659 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:58:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 166 k 2023-01-25T04:58:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:29.660 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:58:29.660 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:58:29.660 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-01-25T04:58:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:29.661 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 166 k 2023-01-25T04:58:29.661 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 503 k 2023-01-25T04:58:29.661 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:58:29.756 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:49 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:29.992 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-immutable-object-cache-18.0.0-1959.gb0d259 498 kB/s | 166 kB 00:00 2023-01-25T04:58:29.993 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:58:29.993 INFO:teuthology.orchestra.run.smithi135.stdout:Total 496 kB/s | 166 kB 00:00 2023-01-25T04:58:29.993 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:58:30.004 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:58:30.005 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:58:30.057 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:58:30.061 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:58:30.398 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:58:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T04:58:30.929 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T04:58:31.017 INFO:teuthology.orchestra.run.smithi178.stdout:Package ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:58:31.122 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:58:31.124 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-01-25T04:58:31.124 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:58:31.236 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T04:58:31.579 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:32.003 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T04:58:32.003 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:52 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:32.441 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:52 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:33.826 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:58:33.827 INFO:teuthology.orchestra.run.smithi178.stdout:=================================================================================================== 2023-01-25T04:58:33.827 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-01-25T04:58:33.828 INFO:teuthology.orchestra.run.smithi178.stdout:=================================================================================================== 2023-01-25T04:58:33.828 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:58:33.828 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 1.7 M 2023-01-25T04:58:33.829 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-01-25T04:58:33.829 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 23 k 2023-01-25T04:58:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 15 k 2023-01-25T04:58:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T04:58:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T04:58:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T04:58:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:33.831 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:58:33.832 INFO:teuthology.orchestra.run.smithi178.stdout:=================================================================================================== 2023-01-25T04:58:33.832 INFO:teuthology.orchestra.run.smithi178.stdout:Install 6 Packages 2023-01-25T04:58:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:33.834 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 2.0 M 2023-01-25T04:58:33.834 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 8.2 M 2023-01-25T04:58:33.834 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:58:33.983 INFO:teuthology.orchestra.run.smithi178.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1959.gb0d 150 kB/s | 23 kB 00:00 2023-01-25T04:58:34.008 INFO:teuthology.orchestra.run.smithi178.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1959.gb0d2 81 kB/s | 15 kB 00:00 2023-01-25T04:58:34.108 INFO:teuthology.orchestra.run.smithi178.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 269 kB/s | 34 kB 00:00 2023-01-25T04:58:34.150 INFO:teuthology.orchestra.run.smithi178.stdout:(4/6): ceph-mgr-dashboard-18.0.0-1959.gb0d25925 5.4 MB/s | 1.7 MB 00:00 2023-01-25T04:58:34.192 INFO:teuthology.orchestra.run.smithi178.stdout:(5/6): python3-jwt-1.6.1-2.el8.noarch.rpm 509 kB/s | 43 kB 00:00 2023-01-25T04:58:34.226 INFO:teuthology.orchestra.run.smithi178.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 904 kB/s | 196 kB 00:00 2023-01-25T04:58:34.227 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:58:34.228 INFO:teuthology.orchestra.run.smithi178.stdout:Total 5.1 MB/s | 2.0 MB 00:00 2023-01-25T04:58:34.239 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:58:34.248 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:58:34.248 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:58:34.304 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:58:34.308 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:58:34.553 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:58:34.835 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:58:35.016 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:35.016 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:58:35.017 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:58:35.017 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:35.017 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:58:35.156 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr 2023-01-25T04:58:35.502 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:35.866 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-25T04:58:36.063 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-25T04:58:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-25T04:58:36.286 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:56 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:36.378 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 4/6 2023-01-25T04:58:36.729 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 5/6 2023-01-25T04:58:37.006 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T04:58:37.222 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T04:58:37.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 1/6 2023-01-25T04:58:37.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 2/6 2023-01-25T04:58:37.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 3/6 2023-01-25T04:58:37.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-25T04:58:37.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-25T04:58:37.571 INFO:teuthology.orchestra.run.smithi135.stdout:Package ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:58:37.683 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:58:37.684 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-01-25T04:58:37.684 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:58:37.795 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T04:58:38.136 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:38.253 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-25T04:58:38.253 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:58 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:38.976 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:58 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:40.377 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:58:40.378 INFO:teuthology.orchestra.run.smithi135.stdout:=================================================================================================== 2023-01-25T04:58:40.378 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-01-25T04:58:40.378 INFO:teuthology.orchestra.run.smithi135.stdout:=================================================================================================== 2023-01-25T04:58:40.379 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:58:40.379 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 1.7 M 2023-01-25T04:58:40.379 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-01-25T04:58:40.379 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 23 k 2023-01-25T04:58:40.379 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 15 k 2023-01-25T04:58:40.380 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T04:58:40.380 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T04:58:40.380 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T04:58:40.380 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:40.380 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:58:40.381 INFO:teuthology.orchestra.run.smithi135.stdout:=================================================================================================== 2023-01-25T04:58:40.381 INFO:teuthology.orchestra.run.smithi135.stdout:Install 6 Packages 2023-01-25T04:58:40.381 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:40.382 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 2.0 M 2023-01-25T04:58:40.382 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 8.2 M 2023-01-25T04:58:40.382 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:58:40.598 INFO:teuthology.orchestra.run.smithi135.stdout:(1/6): ceph-prometheus-alerts-18.0.0-1959.gb0d2 67 kB/s | 15 kB 00:00 2023-01-25T04:58:40.783 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:58:40.876 INFO:teuthology.orchestra.run.smithi135.stdout:(2/6): ceph-grafana-dashboards-18.0.0-1959.gb0d 46 kB/s | 23 kB 00:00 2023-01-25T04:58:40.966 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:40.966 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:58:40.967 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:58:40.967 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:58:40.967 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:58:40.967 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T04:58:40.967 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T04:58:40.968 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T04:58:40.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:40.968 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:58:41.115 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T04:58:41.217 INFO:teuthology.orchestra.run.smithi135.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 54 kB/s | 34 kB 00:00 2023-01-25T04:58:41.461 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:41.495 INFO:teuthology.orchestra.run.smithi135.stdout:(4/6): python3-routes-2.4.1-12.el8.noarch.rpm 317 kB/s | 196 kB 00:00 2023-01-25T04:58:41.634 INFO:teuthology.orchestra.run.smithi135.stdout:(5/6): ceph-mgr-dashboard-18.0.0-1959.gb0d25925 1.4 MB/s | 1.7 MB 00:01 2023-01-25T04:58:41.660 INFO:teuthology.orchestra.run.smithi135.stdout:(6/6): python3-jwt-1.6.1-2.el8.noarch.rpm 96 kB/s | 43 kB 00:00 2023-01-25T04:58:41.662 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:58:41.662 INFO:teuthology.orchestra.run.smithi135.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2023-01-25T04:58:41.675 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:58:41.685 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:58:41.685 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:58:41.746 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:58:41.750 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:58:41.994 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:58:42.170 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-25T04:58:42.298 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:02 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:42.358 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-25T04:58:42.588 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-25T04:58:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 4/6 2023-01-25T04:58:43.067 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 5/6 2023-01-25T04:58:43.298 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T04:58:43.535 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T04:58:43.535 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 1/6 2023-01-25T04:58:43.536 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 2/6 2023-01-25T04:58:43.536 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 3/6 2023-01-25T04:58:43.536 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-25T04:58:43.536 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-25T04:58:43.755 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:58:43.757 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================================================= 2023-01-25T04:58:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-01-25T04:58:43.757 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================================================= 2023-01-25T04:58:43.757 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:58:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 7.4 M 2023-01-25T04:58:43.758 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-01-25T04:58:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-01-25T04:58:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-01-25T04:58:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-25T04:58:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-01-25T04:58:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-25T04:58:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-25T04:58:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-01-25T04:58:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:43.760 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:58:43.760 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================================================= 2023-01-25T04:58:43.760 INFO:teuthology.orchestra.run.smithi178.stdout:Install 8 Packages 2023-01-25T04:58:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:43.761 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 35 M 2023-01-25T04:58:43.761 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 210 M 2023-01-25T04:58:43.761 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:58:43.824 INFO:teuthology.orchestra.run.smithi178.stdout:(1/8): libquadmath-8.5.0-10.1.el8_6.x86_64.rpm 2.6 MB/s | 171 kB 00:00 2023-01-25T04:58:43.866 INFO:teuthology.orchestra.run.smithi178.stdout:(2/8): libgfortran-8.5.0-10.1.el8_6.x86_64.rpm 5.9 MB/s | 644 kB 00:00 2023-01-25T04:58:43.967 INFO:teuthology.orchestra.run.smithi178.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 37 MB/s | 3.7 MB 00:00 2023-01-25T04:58:44.009 INFO:teuthology.orchestra.run.smithi178.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 5.3 MB/s | 225 kB 00:00 2023-01-25T04:58:44.142 INFO:teuthology.orchestra.run.smithi178.stdout:(5/8): openblas-threads-0.3.15-3.el8.x86_64.rpm 36 MB/s | 4.9 MB 00:00 2023-01-25T04:58:44.284 INFO:teuthology.orchestra.run.smithi178.stdout:(6/8): python3-scipy-1.0.0-21.module+el8.5.0+10 30 MB/s | 14 MB 00:00 2023-01-25T04:58:44.393 INFO:teuthology.orchestra.run.smithi178.stdout:(7/8): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-01-25T04:58:44.476 INFO:teuthology.orchestra.run.smithi178.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-195 10 MB/s | 7.4 MB 00:00 2023-01-25T04:58:44.477 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:58:44.478 INFO:teuthology.orchestra.run.smithi178.stdout:Total 49 MB/s | 35 MB 00:00 2023-01-25T04:58:44.552 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-25T04:58:44.553 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:04 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:44.715 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:58:44.734 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:58:44.734 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:58:45.026 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:58:45.030 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:58:45.568 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:58:45.703 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T04:58:45.828 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T04:58:45.953 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T04:58:46.560 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T04:58:46.728 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T04:58:46.961 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:58:47.140 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:47.140 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:58:47.140 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:58:47.141 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:58:47.141 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:58:47.141 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T04:58:47.141 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T04:58:47.141 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T04:58:47.142 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:47.142 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:58:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T04:58:47.318 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T04:58:47.486 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T04:58:47.665 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:48.171 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T04:58:48.464 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:08 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:48.497 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T04:58:48.677 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T04:58:49.927 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:58:49.928 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================================================= 2023-01-25T04:58:49.928 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-01-25T04:58:49.929 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================================================= 2023-01-25T04:58:49.929 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:58:49.929 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 7.4 M 2023-01-25T04:58:49.929 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-01-25T04:58:49.929 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-01-25T04:58:49.930 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-01-25T04:58:49.930 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-25T04:58:49.930 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-01-25T04:58:49.930 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-25T04:58:49.930 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-25T04:58:49.931 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-01-25T04:58:49.931 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:49.931 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:58:49.931 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================================================= 2023-01-25T04:58:49.931 INFO:teuthology.orchestra.run.smithi135.stdout:Install 8 Packages 2023-01-25T04:58:49.932 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:58:49.932 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 35 M 2023-01-25T04:58:49.932 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 210 M 2023-01-25T04:58:49.933 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:58:50.001 INFO:teuthology.orchestra.run.smithi135.stdout:(1/8): libgfortran-8.5.0-10.1.el8_6.x86_64.rpm 9.0 MB/s | 644 kB 00:00 2023-01-25T04:58:50.053 INFO:teuthology.orchestra.run.smithi135.stdout:(2/8): libquadmath-8.5.0-10.1.el8_6.x86_64.rpm 1.4 MB/s | 171 kB 00:00 2023-01-25T04:58:50.169 INFO:teuthology.orchestra.run.smithi135.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 32 MB/s | 3.7 MB 00:00 2023-01-25T04:58:50.210 INFO:teuthology.orchestra.run.smithi135.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 5.3 MB/s | 225 kB 00:00 2023-01-25T04:58:50.345 INFO:teuthology.orchestra.run.smithi135.stdout:(5/8): openblas-threads-0.3.15-3.el8.x86_64.rpm 36 MB/s | 4.9 MB 00:00 2023-01-25T04:58:50.424 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T04:58:50.495 INFO:teuthology.orchestra.run.smithi135.stdout:(6/8): python3-scipy-1.0.0-21.module+el8.5.0+10 28 MB/s | 14 MB 00:00 2023-01-25T04:58:50.612 INFO:teuthology.orchestra.run.smithi135.stdout:(7/8): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-01-25T04:58:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 7/8 2023-01-25T04:58:51.753 INFO:teuthology.orchestra.run.smithi135.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-195 4.1 MB/s | 7.4 MB 00:01 2023-01-25T04:58:51.755 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:58:51.755 INFO:teuthology.orchestra.run.smithi135.stdout:Total 19 MB/s | 35 MB 00:01 2023-01-25T04:58:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T04:58:51.997 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:58:52.016 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:58:52.016 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:58:52.069 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T04:58:52.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 1/8 2023-01-25T04:58:52.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T04:58:52.070 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 3/8 2023-01-25T04:58:52.070 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 4/8 2023-01-25T04:58:52.070 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T04:58:52.070 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T04:58:52.070 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 7/8 2023-01-25T04:58:52.307 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:58:52.311 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:58:52.896 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:58:53.076 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T04:58:53.109 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 8/8 2023-01-25T04:58:53.109 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:13 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:53.212 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T04:58:53.376 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T04:58:53.993 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T04:58:54.184 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T04:58:54.777 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T04:58:54.984 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T04:58:55.568 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:58:55.667 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T04:58:55.750 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:55.751 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:58:55.751 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:58:55.751 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-01-25T04:58:55.751 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-01-25T04:58:55.751 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.15-3.el8.x86_64 2023-01-25T04:58:55.752 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-01-25T04:58:55.752 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T04:58:55.752 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T04:58:55.752 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-01-25T04:58:55.752 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:55.753 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:58:55.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-rook 2023-01-25T04:58:56.037 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T04:58:56.235 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:58:56.284 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T04:58:57.122 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:17 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:58:58.034 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T04:58:58.524 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:58:58.526 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================================== 2023-01-25T04:58:58.526 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-01-25T04:58:58.526 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================================== 2023-01-25T04:58:58.527 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:58:58.527 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 47 k 2023-01-25T04:58:58.527 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-01-25T04:58:58.527 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T04:58:58.527 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T04:58:58.528 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T04:58:58.528 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-25T04:58:58.528 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-25T04:58:58.528 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T04:58:58.528 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-25T04:58:58.529 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-25T04:58:58.529 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-25T04:58:58.529 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T04:58:58.529 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T04:58:58.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T04:58:58.530 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:58.530 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:58:58.530 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================================== 2023-01-25T04:58:58.531 INFO:teuthology.orchestra.run.smithi178.stdout:Install 13 Packages 2023-01-25T04:58:58.531 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:58:58.531 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 1.9 M 2023-01-25T04:58:58.532 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 25 M 2023-01-25T04:58:58.532 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:58:58.648 INFO:teuthology.orchestra.run.smithi178.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 138 kB/s | 16 kB 00:00 2023-01-25T04:58:58.732 INFO:teuthology.orchestra.run.smithi178.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 163 kB/s | 33 kB 00:00 2023-01-25T04:58:58.932 INFO:teuthology.orchestra.run.smithi178.stdout:(3/13): python3-kubernetes-11.0.0-6.el8.noarch. 5.8 MB/s | 1.2 MB 00:00 2023-01-25T04:58:58.982 INFO:teuthology.orchestra.run.smithi178.stdout:(4/13): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2023-01-25T04:58:59.009 INFO:teuthology.orchestra.run.smithi178.stdout:(5/13): ceph-mgr-rook-18.0.0-1959.gb0d25925.el8 99 kB/s | 47 kB 00:00 2023-01-25T04:58:59.033 INFO:teuthology.orchestra.run.smithi178.stdout:(6/13): python3-websocket-client-0.56.0-5.el8.n 1.2 MB/s | 61 kB 00:00 2023-01-25T04:58:59.083 INFO:teuthology.orchestra.run.smithi178.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 2.0 MB/s | 155 kB 00:00 2023-01-25T04:58:59.108 INFO:teuthology.orchestra.run.smithi178.stdout:(8/13): python3-requests-oauthlib-1.0.0-1.el8.n 609 kB/s | 43 kB 00:00 2023-01-25T04:58:59.133 INFO:teuthology.orchestra.run.smithi178.stdout:(9/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-01-25T04:58:59.159 INFO:teuthology.orchestra.run.smithi178.stdout:(10/13): python3-jsonpatch-1.21-2.el8.noarch.rp 528 kB/s | 27 kB 00:00 2023-01-25T04:58:59.176 INFO:teuthology.orchestra.run.smithi178.stdout:(11/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.9 MB/s | 126 kB 00:00 2023-01-25T04:58:59.192 INFO:teuthology.orchestra.run.smithi178.stdout:(12/13): python3-jsonpointer-1.10-11.el8.noarch 602 kB/s | 20 kB 00:00 2023-01-25T04:58:59.304 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 7/8 2023-01-25T04:58:59.458 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T04:58:59.735 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T04:58:59.735 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 1/8 2023-01-25T04:58:59.735 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T04:58:59.735 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 3/8 2023-01-25T04:58:59.736 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 4/8 2023-01-25T04:58:59.736 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T04:58:59.736 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T04:58:59.736 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 7/8 2023-01-25T04:58:59.993 INFO:teuthology.orchestra.run.smithi178.stdout:(13/13): python3-google-auth-1.1.1-10.el8.noarc 60 kB/s | 81 kB 00:01 2023-01-25T04:58:59.993 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:58:59.994 INFO:teuthology.orchestra.run.smithi178.stdout:Total 1.3 MB/s | 1.9 MB 00:01 2023-01-25T04:59:00.060 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:59:00.076 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:59:00.076 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:59:00.147 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:59:00.151 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:59:00.620 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:59:00.808 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T04:59:00.809 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 8/8 2023-01-25T04:59:00.810 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:20 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:01.027 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-25T04:59:01.189 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-25T04:59:01.332 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-25T04:59:01.528 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-25T04:59:01.737 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T04:59:01.977 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T04:59:02.165 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-25T04:59:02.347 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T04:59:02.555 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-25T04:59:03.511 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:59:03.644 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-25T04:59:03.694 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:03.694 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:59:03.694 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:59:03.694 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-01-25T04:59:03.694 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-01-25T04:59:03.695 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.15-3.el8.x86_64 2023-01-25T04:59:03.695 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-01-25T04:59:03.695 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T04:59:03.695 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T04:59:03.695 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-01-25T04:59:03.696 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:03.696 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:59:03.835 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-rook 2023-01-25T04:59:03.900 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-25T04:59:04.041 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T04:59:04.181 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:04.280 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T04:59:04.280 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 1/13 2023-01-25T04:59:04.281 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-25T04:59:04.281 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-25T04:59:04.281 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-25T04:59:04.282 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-25T04:59:04.282 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-25T04:59:04.282 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-25T04:59:04.283 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T04:59:04.283 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T04:59:04.283 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-01-25T04:59:04.284 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 11/13 2023-01-25T04:59:04.284 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 12/13 2023-01-25T04:59:05.050 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:25 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:05.356 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-25T04:59:05.356 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:25 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:06.424 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:59:06.426 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================================== 2023-01-25T04:59:06.426 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-01-25T04:59:06.426 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================================== 2023-01-25T04:59:06.426 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:59:06.427 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 47 k 2023-01-25T04:59:06.427 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-01-25T04:59:06.427 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T04:59:06.427 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T04:59:06.427 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T04:59:06.428 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-25T04:59:06.428 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-25T04:59:06.428 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T04:59:06.428 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-25T04:59:06.428 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-25T04:59:06.429 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-25T04:59:06.429 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T04:59:06.429 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T04:59:06.429 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T04:59:06.429 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:06.429 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:59:06.430 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================================== 2023-01-25T04:59:06.430 INFO:teuthology.orchestra.run.smithi135.stdout:Install 13 Packages 2023-01-25T04:59:06.430 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:06.431 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 1.9 M 2023-01-25T04:59:06.431 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 25 M 2023-01-25T04:59:06.431 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:59:06.649 INFO:teuthology.orchestra.run.smithi135.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 74 kB/s | 16 kB 00:00 2023-01-25T04:59:06.674 INFO:teuthology.orchestra.run.smithi135.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 134 kB/s | 33 kB 00:00 2023-01-25T04:59:06.733 INFO:teuthology.orchestra.run.smithi135.stdout:(3/13): ceph-mgr-rook-18.0.0-1959.gb0d25925.el8 156 kB/s | 47 kB 00:00 2023-01-25T04:59:06.758 INFO:teuthology.orchestra.run.smithi135.stdout:(4/13): python3-google-auth-1.1.1-10.el8.noarch 745 kB/s | 81 kB 00:00 2023-01-25T04:59:06.784 INFO:teuthology.orchestra.run.smithi135.stdout:(5/13): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2023-01-25T04:59:06.851 INFO:teuthology.orchestra.run.smithi135.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 2.3 MB/s | 155 kB 00:00 2023-01-25T04:59:06.901 INFO:teuthology.orchestra.run.smithi135.stdout:(7/13): python3-kubernetes-11.0.0-6.el8.noarch. 5.1 MB/s | 1.2 MB 00:00 2023-01-25T04:59:06.927 INFO:teuthology.orchestra.run.smithi135.stdout:(8/13): python3-requests-oauthlib-1.0.0-1.el8.n 564 kB/s | 43 kB 00:00 2023-01-25T04:59:06.961 INFO:teuthology.orchestra.run.smithi135.stdout:(9/13): python3-websocket-client-0.56.0-5.el8.n 303 kB/s | 61 kB 00:00 2023-01-25T04:59:06.986 INFO:teuthology.orchestra.run.smithi135.stdout:(10/13): python3-pyasn1-modules-0.3.7-6.el8.noa 1.3 MB/s | 110 kB 00:00 2023-01-25T04:59:07.011 INFO:teuthology.orchestra.run.smithi135.stdout:(11/13): python3-jsonpatch-1.21-2.el8.noarch.rp 337 kB/s | 27 kB 00:00 2023-01-25T04:59:07.036 INFO:teuthology.orchestra.run.smithi135.stdout:(12/13): python3-jsonpointer-1.10-11.el8.noarch 445 kB/s | 20 kB 00:00 2023-01-25T04:59:07.062 INFO:teuthology.orchestra.run.smithi135.stdout:(13/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.2 MB/s | 126 kB 00:00 2023-01-25T04:59:07.063 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:59:07.063 INFO:teuthology.orchestra.run.smithi135.stdout:Total 3.0 MB/s | 1.9 MB 00:00 2023-01-25T04:59:07.120 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:59:07.135 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:59:07.135 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:59:07.205 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:59:07.209 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:59:07.707 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:59:07.743 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:59:07.927 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:07.927 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:59:07.927 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:59:07.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T04:59:07.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T04:59:07.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T04:59:07.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T04:59:07.929 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T04:59:07.929 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T04:59:07.929 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T04:59:07.929 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T04:59:07.929 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T04:59:07.930 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T04:59:07.930 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T04:59:07.930 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T04:59:07.930 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:07.930 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:59:07.949 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T04:59:08.062 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T04:59:08.226 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-25T04:59:08.414 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:08.422 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-25T04:59:08.639 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-25T04:59:08.878 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-25T04:59:09.170 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T04:59:09.247 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:29 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:09.394 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T04:59:09.631 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-25T04:59:09.805 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T04:59:10.030 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-25T04:59:10.702 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:59:10.703 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================================== 2023-01-25T04:59:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-01-25T04:59:10.703 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================================== 2023-01-25T04:59:10.703 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:59:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 120 k 2023-01-25T04:59:10.704 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-01-25T04:59:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T04:59:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-25T04:59:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-25T04:59:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T04:59:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:10.705 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:59:10.706 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================================== 2023-01-25T04:59:10.706 INFO:teuthology.orchestra.run.smithi178.stdout:Install 5 Packages 2023-01-25T04:59:10.706 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:10.707 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 6.0 M 2023-01-25T04:59:10.707 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 26 M 2023-01-25T04:59:10.707 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:59:10.858 INFO:teuthology.orchestra.run.smithi178.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 395 kB/s | 60 kB 00:00 2023-01-25T04:59:10.941 INFO:teuthology.orchestra.run.smithi178.stdout:(2/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 6.4 MB/s | 538 kB 00:00 2023-01-25T04:59:10.967 INFO:teuthology.orchestra.run.smithi178.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.8 MB/s | 476 kB 00:00 2023-01-25T04:59:11.059 INFO:teuthology.orchestra.run.smithi178.stdout:(4/5): python3-babel-2.5.1-7.el8.noarch.rpm 41 MB/s | 4.8 MB 00:00 2023-01-25T04:59:11.139 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-25T04:59:11.334 INFO:teuthology.orchestra.run.smithi178.stdout:(5/5): ceph-mgr-cephadm-18.0.0-1959.gb0d25925.e 191 kB/s | 120 kB 00:00 2023-01-25T04:59:11.335 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:59:11.335 INFO:teuthology.orchestra.run.smithi178.stdout:Total 9.4 MB/s | 6.0 MB 00:00 2023-01-25T04:59:11.388 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:59:11.398 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-25T04:59:11.399 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:59:11.399 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:59:11.477 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:59:11.488 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:59:11.573 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T04:59:11.815 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T04:59:11.815 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 1/13 2023-01-25T04:59:11.816 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-25T04:59:11.816 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-25T04:59:11.816 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-25T04:59:11.817 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-25T04:59:11.817 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-25T04:59:11.817 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-25T04:59:11.817 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T04:59:11.817 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T04:59:11.818 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-01-25T04:59:11.818 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 11/13 2023-01-25T04:59:11.818 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 12/13 2023-01-25T04:59:12.469 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:59:12.818 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2023-01-25T04:59:12.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-25T04:59:12.932 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:32 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:13.024 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2023-01-25T04:59:13.336 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T04:59:13.651 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T04:59:13.839 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T04:59:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T04:59:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/5 2023-01-25T04:59:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T04:59:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T04:59:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T04:59:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 5/5 2023-01-25T04:59:15.092 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:35 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:16.124 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:59:16.306 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:16.306 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:59:16.306 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:59:16.306 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T04:59:16.307 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T04:59:16.307 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T04:59:16.307 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T04:59:16.307 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T04:59:16.308 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T04:59:16.308 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T04:59:16.308 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T04:59:16.308 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T04:59:16.308 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T04:59:16.309 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T04:59:16.309 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T04:59:16.309 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:16.309 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:59:16.451 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T04:59:16.805 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:17.701 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:37 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:18.343 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:59:18.525 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:18.525 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:59:18.525 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:59:18.526 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T04:59:18.526 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T04:59:18.526 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T04:59:18.526 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T04:59:18.526 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:18.527 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:59:18.642 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-fuse 2023-01-25T04:59:18.994 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:19.100 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:59:19.104 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================================== 2023-01-25T04:59:19.105 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-01-25T04:59:19.105 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================================== 2023-01-25T04:59:19.105 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:59:19.105 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 120 k 2023-01-25T04:59:19.106 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-01-25T04:59:19.106 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T04:59:19.106 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-25T04:59:19.107 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-25T04:59:19.107 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T04:59:19.107 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:19.107 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:59:19.107 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================================== 2023-01-25T04:59:19.108 INFO:teuthology.orchestra.run.smithi135.stdout:Install 5 Packages 2023-01-25T04:59:19.108 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:19.108 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 6.0 M 2023-01-25T04:59:19.108 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 26 M 2023-01-25T04:59:19.108 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:59:19.248 INFO:teuthology.orchestra.run.smithi135.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 416 kB/s | 60 kB 00:00 2023-01-25T04:59:19.324 INFO:teuthology.orchestra.run.smithi135.stdout:(2/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 7.0 MB/s | 538 kB 00:00 2023-01-25T04:59:19.349 INFO:teuthology.orchestra.run.smithi135.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.9 MB/s | 476 kB 00:00 2023-01-25T04:59:19.468 INFO:teuthology.orchestra.run.smithi135.stdout:(4/5): python3-babel-2.5.1-7.el8.noarch.rpm 33 MB/s | 4.8 MB 00:00 2023-01-25T04:59:19.641 INFO:teuthology.orchestra.run.smithi135.stdout:(5/5): ceph-mgr-cephadm-18.0.0-1959.gb0d25925.e 223 kB/s | 120 kB 00:00 2023-01-25T04:59:19.642 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:59:19.642 INFO:teuthology.orchestra.run.smithi135.stdout:Total 11 MB/s | 6.0 MB 00:00 2023-01-25T04:59:19.695 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:59:19.707 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:59:19.707 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:59:19.784 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:59:19.788 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:59:19.881 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:39 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:20.776 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:59:21.190 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2023-01-25T04:59:21.265 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:59:21.266 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-01-25T04:59:21.266 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:21.266 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:59:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse 2023-01-25T04:59:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 845 k 2023-01-25T04:59:21.267 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-01-25T04:59:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-25T04:59:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:21.268 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:59:21.268 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:21.268 INFO:teuthology.orchestra.run.smithi178.stdout:Install 2 Packages 2023-01-25T04:59:21.269 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:21.269 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 927 k 2023-01-25T04:59:21.270 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 2.9 M 2023-01-25T04:59:21.270 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:59:21.351 INFO:teuthology.orchestra.run.smithi178.stdout:(1/2): fuse-2.9.7-15.el8.x86_64.rpm 1.0 MB/s | 83 kB 00:00 2023-01-25T04:59:21.414 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2023-01-25T04:59:21.749 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T04:59:22.049 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T04:59:22.246 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T04:59:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T04:59:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/5 2023-01-25T04:59:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T04:59:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T04:59:22.506 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T04:59:22.509 INFO:teuthology.orchestra.run.smithi178.stdout:(2/2): ceph-fuse-18.0.0-1959.gb0d25925.el8.x86_ 681 kB/s | 845 kB 00:01 2023-01-25T04:59:22.509 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:59:22.510 INFO:teuthology.orchestra.run.smithi178.stdout:Total 746 kB/s | 927 kB 00:01 2023-01-25T04:59:22.522 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:59:22.545 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:59:22.545 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:59:22.621 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:59:22.625 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:59:22.783 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:59:23.044 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.7-15.el8.x86_64 1/2 2023-01-25T04:59:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T04:59:23.521 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 5/5 2023-01-25T04:59:23.521 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:43 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T04:59:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/2 2023-01-25T04:59:25.275 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 2/2 2023-01-25T04:59:25.275 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:45 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:26.252 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:59:26.436 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:26.436 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:59:26.437 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:59:26.437 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T04:59:26.437 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T04:59:26.437 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T04:59:26.437 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T04:59:26.438 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:26.438 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:59:26.554 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-fuse 2023-01-25T04:59:26.906 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:27.759 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:47 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:28.454 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:59:28.634 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:28.635 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:59:28.635 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 fuse-2.9.7-15.el8.x86_64 2023-01-25T04:59:28.635 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:28.636 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:59:28.760 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-volume 2023-01-25T04:59:29.116 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:29.215 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:59:29.215 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:29.216 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-01-25T04:59:29.216 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:29.216 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:59:29.216 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse 2023-01-25T04:59:29.216 INFO:teuthology.orchestra.run.smithi135.stdout: x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 845 k 2023-01-25T04:59:29.217 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-01-25T04:59:29.217 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-25T04:59:29.217 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:29.217 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:59:29.217 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:29.218 INFO:teuthology.orchestra.run.smithi135.stdout:Install 2 Packages 2023-01-25T04:59:29.218 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:29.219 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 927 k 2023-01-25T04:59:29.219 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 2.9 M 2023-01-25T04:59:29.220 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:59:29.914 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:49 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:30.165 INFO:teuthology.orchestra.run.smithi135.stdout:(1/2): fuse-2.9.7-15.el8.x86_64.rpm 87 kB/s | 83 kB 00:00 2023-01-25T04:59:30.798 INFO:teuthology.orchestra.run.smithi135.stdout:(2/2): ceph-fuse-18.0.0-1959.gb0d25925.el8.x86_ 534 kB/s | 845 kB 00:01 2023-01-25T04:59:30.799 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:59:30.799 INFO:teuthology.orchestra.run.smithi135.stdout:Total 586 kB/s | 927 kB 00:01 2023-01-25T04:59:30.812 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:59:30.835 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:59:30.835 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:59:30.914 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:59:30.917 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:59:31.078 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:59:31.306 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:59:31.307 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:31.308 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-01-25T04:59:31.308 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:31.308 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:59:31.308 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 263 k 2023-01-25T04:59:31.309 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:31.309 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:59:31.309 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:31.309 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2023-01-25T04:59:31.309 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:31.310 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 263 k 2023-01-25T04:59:31.310 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 1.3 M 2023-01-25T04:59:31.310 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:59:31.368 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.7-15.el8.x86_64 1/2 2023-01-25T04:59:31.535 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T04:59:31.862 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T04:59:31.862 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/2 2023-01-25T04:59:31.864 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-volume-18.0.0-1959.gb0d25925.el8.noarch.rp 473 kB/s | 263 kB 00:00 2023-01-25T04:59:31.865 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:59:31.866 INFO:teuthology.orchestra.run.smithi178.stdout:Total 471 kB/s | 263 kB 00:00 2023-01-25T04:59:31.866 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:59:31.876 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:59:31.876 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:59:31.893 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:59:31.897 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:59:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:59:32.278 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:59:32.629 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:59:32.952 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 2/2 2023-01-25T04:59:32.952 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:52 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:33.676 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:59:33.676 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:53 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:35.833 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:59:36.014 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:36.014 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:59:36.014 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 fuse-2.9.7-15.el8.x86_64 2023-01-25T04:59:36.015 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:36.015 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:59:36.142 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-volume 2023-01-25T04:59:36.500 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:36.671 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:59:36.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:36.852 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:59:36.852 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:59:36.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:36.853 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:59:36.977 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librados-devel 2023-01-25T04:59:37.331 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:37.335 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:57 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:38.223 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:58 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:38.705 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:59:38.706 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:38.706 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-01-25T04:59:38.707 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:38.707 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:59:38.707 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 263 k 2023-01-25T04:59:38.707 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:38.707 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:59:38.708 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:38.708 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-01-25T04:59:38.708 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:38.708 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 263 k 2023-01-25T04:59:38.708 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 1.3 M 2023-01-25T04:59:38.709 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:59:39.081 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-volume-18.0.0-1959.gb0d25925.el8.noarch.rp 704 kB/s | 263 kB 00:00 2023-01-25T04:59:39.082 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:59:39.082 INFO:teuthology.orchestra.run.smithi135.stdout:Total 702 kB/s | 263 kB 00:00 2023-01-25T04:59:39.082 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:59:39.092 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:59:39.092 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:59:39.109 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:59:39.112 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:59:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:59:39.553 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:59:39.675 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:59:39.676 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:39.676 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2023-01-25T04:59:39.676 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:39.677 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:59:39.677 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 139 k 2023-01-25T04:59:39.677 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:39.677 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:59:39.677 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:39.678 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2023-01-25T04:59:39.678 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:39.678 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 139 k 2023-01-25T04:59:39.678 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 512 k 2023-01-25T04:59:39.679 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:59:39.903 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:59:40.047 INFO:teuthology.orchestra.run.smithi178.stdout:librados-devel-18.0.0-1959.gb0d25925.el8.x86_64 376 kB/s | 139 kB 00:00 2023-01-25T04:59:40.047 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:59:40.048 INFO:teuthology.orchestra.run.smithi178.stdout:Total 375 kB/s | 139 kB 00:00 2023-01-25T04:59:40.048 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:59:40.053 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:59:40.054 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:59:40.108 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:59:40.112 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:59:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:59:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:40.662 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:40.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:59:40.936 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:00 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:41.940 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:41.941 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:01 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:43.871 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:59:44.051 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:44.052 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:59:44.052 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:59:44.052 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:44.053 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:59:44.184 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librados-devel 2023-01-25T04:59:44.538 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:44.759 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:59:44.938 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:44.939 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:59:44.939 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:59:44.939 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:44.940 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:59:45.065 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install libcephfs2 2023-01-25T04:59:45.371 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:05 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:45.422 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:46.351 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:06 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:46.765 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:59:46.766 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:46.767 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2023-01-25T04:59:46.767 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:46.767 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:59:46.767 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 139 k 2023-01-25T04:59:46.767 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:46.768 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:59:46.768 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:46.768 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-01-25T04:59:46.768 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:46.768 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 139 k 2023-01-25T04:59:46.769 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 512 k 2023-01-25T04:59:46.769 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:59:47.351 INFO:teuthology.orchestra.run.smithi135.stdout:librados-devel-18.0.0-1959.gb0d25925.el8.x86_64 238 kB/s | 139 kB 00:00 2023-01-25T04:59:47.352 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:59:47.352 INFO:teuthology.orchestra.run.smithi135.stdout:Total 238 kB/s | 139 kB 00:00 2023-01-25T04:59:47.352 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:59:47.358 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:59:47.358 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:59:47.414 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:59:47.418 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:59:47.535 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:59:47.667 INFO:teuthology.orchestra.run.smithi178.stdout:Package libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:59:47.707 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:47.775 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:59:47.776 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-01-25T04:59:47.776 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:59:47.881 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install libcephfs-devel 2023-01-25T04:59:47.965 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:48.233 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:49.058 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:49.059 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:09 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:49.107 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:09 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:50.495 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:59:50.496 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:50.496 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2023-01-25T04:59:50.497 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:50.497 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T04:59:50.497 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 30 k 2023-01-25T04:59:50.497 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:50.498 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T04:59:50.498 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T04:59:50.498 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2023-01-25T04:59:50.498 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:50.498 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 30 k 2023-01-25T04:59:50.498 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 137 k 2023-01-25T04:59:50.499 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T04:59:50.834 INFO:teuthology.orchestra.run.smithi178.stdout:libcephfs-devel-18.0.0-1959.gb0d25925.el8.x86_6 90 kB/s | 30 kB 00:00 2023-01-25T04:59:50.835 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:59:50.835 INFO:teuthology.orchestra.run.smithi178.stdout:Total 90 kB/s | 30 kB 00:00 2023-01-25T04:59:50.835 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T04:59:50.838 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T04:59:50.838 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T04:59:50.849 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T04:59:50.852 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T04:59:50.908 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T04:59:51.079 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:51.296 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:52.001 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T04:59:52.181 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:52.182 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T04:59:52.182 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:59:52.182 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:52.182 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:59:52.255 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:52.255 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:12 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:52.315 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install libcephfs2 2023-01-25T04:59:52.667 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:53.583 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:13 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:54.849 INFO:teuthology.orchestra.run.smithi135.stdout:Package libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:59:54.960 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:59:54.961 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-01-25T04:59:54.961 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T04:59:55.078 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install libcephfs-devel 2023-01-25T04:59:55.228 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T04:59:55.407 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:55.408 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T04:59:55.408 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:59:55.408 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T04:59:55.408 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:59:55.428 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:55.535 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librados2 2023-01-25T04:59:55.887 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:56.292 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:16 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:56.899 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:16 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:57.658 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T04:59:57.659 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:57.659 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2023-01-25T04:59:57.659 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:57.660 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T04:59:57.660 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 30 k 2023-01-25T04:59:57.660 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:57.660 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T04:59:57.660 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T04:59:57.661 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-01-25T04:59:57.661 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T04:59:57.661 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 30 k 2023-01-25T04:59:57.661 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 137 k 2023-01-25T04:59:57.661 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T04:59:57.826 INFO:teuthology.orchestra.run.smithi135.stdout:libcephfs-devel-18.0.0-1959.gb0d25925.el8.x86_6 183 kB/s | 30 kB 00:00 2023-01-25T04:59:57.827 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:59:57.827 INFO:teuthology.orchestra.run.smithi135.stdout:Total 182 kB/s | 30 kB 00:00 2023-01-25T04:59:57.828 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T04:59:57.831 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T04:59:57.831 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T04:59:57.841 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T04:59:57.845 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T04:59:57.902 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T04:59:58.105 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:58.248 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:59:58.331 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:58.356 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T04:59:58.357 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-01-25T04:59:58.358 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T04:59:58.478 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librbd1 2023-01-25T04:59:58.830 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T04:59:59.339 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:59:59.340 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:19 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T04:59:59.773 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:19 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:01.040 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:00:01.152 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T05:00:01.153 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-01-25T05:00:01.154 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T05:00:01.265 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rados 2023-01-25T05:00:01.621 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:03.140 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:23 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:03.533 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T05:00:03.711 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:03.712 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T05:00:03.712 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:00:03.712 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:03.712 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T05:00:03.845 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librados2 2023-01-25T05:00:04.198 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:04.421 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:00:04.531 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T05:00:04.532 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-01-25T05:00:04.532 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T05:00:04.647 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rgw 2023-01-25T05:00:04.999 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:05.082 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:25 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:06.033 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:26 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:06.432 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:00:06.545 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T05:00:06.545 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-01-25T05:00:06.545 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T05:00:06.656 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librbd1 2023-01-25T05:00:07.012 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:07.299 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:00:07.407 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T05:00:07.408 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-01-25T05:00:07.408 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T05:00:07.520 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-cephfs 2023-01-25T05:00:07.873 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:07.893 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:27 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:08.866 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:28 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:09.166 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:00:09.273 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T05:00:09.274 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-01-25T05:00:09.274 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T05:00:09.386 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rados 2023-01-25T05:00:09.738 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:10.133 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:00:10.241 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T05:00:10.242 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-01-25T05:00:10.242 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T05:00:10.355 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rbd 2023-01-25T05:00:10.673 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:30 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:10.708 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:11.671 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:31 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:11.970 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:00:12.078 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T05:00:12.079 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-01-25T05:00:12.079 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T05:00:12.195 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rgw 2023-01-25T05:00:12.552 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:12.938 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:00:13.047 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T05:00:13.048 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-01-25T05:00:13.049 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T05:00:13.170 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-fuse 2023-01-25T05:00:13.478 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:33 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:13.522 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:14.412 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:34 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:14.745 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:00:14.852 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T05:00:14.853 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-01-25T05:00:14.854 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T05:00:14.967 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-cephfs 2023-01-25T05:00:15.318 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:15.803 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T05:00:15.803 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T05:00:15.804 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2023-01-25T05:00:15.804 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T05:00:15.804 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T05:00:15.804 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 94 k 2023-01-25T05:00:15.805 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:15.805 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T05:00:15.805 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T05:00:15.805 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2023-01-25T05:00:15.805 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:15.806 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 94 k 2023-01-25T05:00:15.806 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 280 k 2023-01-25T05:00:15.806 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T05:00:15.993 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-fuse-18.0.0-1959.gb0d25925.el8.x86_64.rpm 501 kB/s | 94 kB 00:00 2023-01-25T05:00:15.994 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:00:15.994 INFO:teuthology.orchestra.run.smithi178.stdout:Total 498 kB/s | 94 kB 00:00 2023-01-25T05:00:15.994 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T05:00:16.004 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T05:00:16.005 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T05:00:16.057 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T05:00:16.061 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T05:00:16.180 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:36 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:16.183 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T05:00:16.333 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:16.564 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:17.527 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:00:17.626 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:17.626 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:37 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:17.636 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T05:00:17.637 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-01-25T05:00:17.638 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T05:00:17.745 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rbd 2023-01-25T05:00:18.098 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:18.896 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:38 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:20.178 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T05:00:20.285 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T05:00:20.287 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-01-25T05:00:20.287 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T05:00:20.401 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-fuse 2023-01-25T05:00:20.455 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T05:00:20.635 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:20.636 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T05:00:20.636 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:00:20.636 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:20.636 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T05:00:20.753 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:20.760 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-mirror 2023-01-25T05:00:21.112 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:21.549 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:41 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:21.906 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:41 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:22.938 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T05:00:22.939 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T05:00:22.939 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2023-01-25T05:00:22.940 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T05:00:22.940 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T05:00:22.940 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 94 k 2023-01-25T05:00:22.940 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:22.940 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T05:00:22.941 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T05:00:22.941 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-01-25T05:00:22.941 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:22.941 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 94 k 2023-01-25T05:00:22.941 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 280 k 2023-01-25T05:00:22.942 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T05:00:23.147 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-fuse-18.0.0-1959.gb0d25925.el8.x86_64.rpm 456 kB/s | 94 kB 00:00 2023-01-25T05:00:23.148 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:00:23.148 INFO:teuthology.orchestra.run.smithi135.stdout:Total 453 kB/s | 94 kB 00:00 2023-01-25T05:00:23.148 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T05:00:23.158 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T05:00:23.158 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T05:00:23.211 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T05:00:23.215 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T05:00:23.282 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T05:00:23.283 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T05:00:23.283 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2023-01-25T05:00:23.283 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T05:00:23.283 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T05:00:23.283 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.4 M 2023-01-25T05:00:23.284 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:23.284 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T05:00:23.284 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T05:00:23.284 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2023-01-25T05:00:23.285 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:23.285 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 3.4 M 2023-01-25T05:00:23.285 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 16 M 2023-01-25T05:00:23.285 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T05:00:23.331 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T05:00:23.501 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:23.729 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-mirror-18.0.0-1959.gb0d25925.el8.x86_64.rpm 7.8 MB/s | 3.4 MB 00:00 2023-01-25T05:00:23.729 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:00:23.729 INFO:teuthology.orchestra.run.smithi178.stdout:Total 7.7 MB/s | 3.4 MB 00:00 2023-01-25T05:00:23.730 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T05:00:23.737 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:23.741 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T05:00:23.742 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T05:00:23.808 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T05:00:23.812 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T05:00:24.250 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T05:00:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:24.783 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:24.810 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:24.811 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:44 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:25.831 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:25.831 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:45 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:27.535 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T05:00:27.714 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:27.714 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T05:00:27.714 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:00:27.714 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:27.715 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T05:00:27.845 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-mirror 2023-01-25T05:00:28.197 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:29.125 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:49 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:29.336 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T05:00:29.516 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:29.516 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T05:00:29.516 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:00:29.517 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:29.517 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T05:00:29.646 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-nbd 2023-01-25T05:00:30.000 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:30.508 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T05:00:30.509 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T05:00:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2023-01-25T05:00:30.510 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T05:00:30.510 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T05:00:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.4 M 2023-01-25T05:00:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:30.511 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T05:00:30.511 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T05:00:30.511 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-01-25T05:00:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:30.512 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 3.4 M 2023-01-25T05:00:30.512 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 16 M 2023-01-25T05:00:30.512 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T05:00:30.943 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:50 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:30.995 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-mirror-18.0.0-1959.gb0d25925.el8.x86_64.rpm 7.1 MB/s | 3.4 MB 00:00 2023-01-25T05:00:30.996 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:00:30.997 INFO:teuthology.orchestra.run.smithi135.stdout:Total 7.1 MB/s | 3.4 MB 00:00 2023-01-25T05:00:30.997 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T05:00:31.008 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T05:00:31.008 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T05:00:31.074 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T05:00:31.078 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T05:00:31.517 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T05:00:31.737 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:32.099 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:32.326 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-01-25T05:00:32.327 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T05:00:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2023-01-25T05:00:32.327 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T05:00:32.328 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-01-25T05:00:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 179 k 2023-01-25T05:00:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:32.328 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-01-25T05:00:32.328 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-01-25T05:00:32.329 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2023-01-25T05:00:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:32.329 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 179 k 2023-01-25T05:00:32.329 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 568 k 2023-01-25T05:00:32.329 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-01-25T05:00:32.539 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-nbd-18.0.0-1959.gb0d25925.el8.x86_64.rpm 849 kB/s | 179 kB 00:00 2023-01-25T05:00:32.540 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:00:32.541 INFO:teuthology.orchestra.run.smithi178.stdout:Total 844 kB/s | 179 kB 00:00 2023-01-25T05:00:32.541 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-01-25T05:00:32.552 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-01-25T05:00:32.552 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-01-25T05:00:32.606 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-01-25T05:00:32.609 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-01-25T05:00:32.727 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-01-25T05:00:32.872 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:33.095 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:33.135 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:33.135 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:53 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:34.130 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:34.130 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:03:54 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:35.552 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T05:00:35.736 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:35.736 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T05:00:35.736 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:00:35.737 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:35.737 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T05:00:35.873 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-nbd 2023-01-25T05:00:36.226 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-01-25T05:00:36.517 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-01-25T05:00:36.697 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:36.697 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-01-25T05:00:36.697 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:00:36.698 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:36.698 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-01-25T05:00:36.826 DEBUG:teuthology.parallel:result is None 2023-01-25T05:00:36.997 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:03:56 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:38.395 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-01-25T05:00:38.396 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T05:00:38.397 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2023-01-25T05:00:38.398 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T05:00:38.398 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-01-25T05:00:38.399 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 179 k 2023-01-25T05:00:38.399 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:38.399 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-01-25T05:00:38.399 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-01-25T05:00:38.399 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-01-25T05:00:38.400 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:38.400 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 179 k 2023-01-25T05:00:38.400 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 568 k 2023-01-25T05:00:38.400 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-01-25T05:00:38.593 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-nbd-18.0.0-1959.gb0d25925.el8.x86_64.rpm 916 kB/s | 179 kB 00:00 2023-01-25T05:00:38.594 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-01-25T05:00:38.594 INFO:teuthology.orchestra.run.smithi135.stdout:Total 910 kB/s | 179 kB 00:00 2023-01-25T05:00:38.595 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-01-25T05:00:38.606 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-01-25T05:00:38.606 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-01-25T05:00:38.661 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-01-25T05:00:38.664 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-01-25T05:00:38.788 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-01-25T05:00:38.983 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:40.299 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T05:00:40.300 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:04:00 ago on Wed 25 Jan 2023 04:56:40 AM UTC. 2023-01-25T05:00:43.852 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-01-25T05:00:44.034 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:44.034 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-01-25T05:00:44.035 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T05:00:44.035 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:44.035 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-01-25T05:00:44.166 DEBUG:teuthology.parallel:result is None 2023-01-25T05:00:44.167 DEBUG:teuthology.orchestra.run.smithi135:> sudo lsb_release -is 2023-01-25T05:00:44.208 INFO:teuthology.orchestra.run.smithi135.stdout:RedHatEnterprise 2023-01-25T05:00:44.210 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:00:44.210 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:00:44.380 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T05:00:44.402 INFO:teuthology.orchestra.run.smithi135.stdout:18.0.0-1959.gb0d25925.el8 2023-01-25T05:00:44.403 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1959.gb0d25925.el8 2023-01-25T05:00:44.403 INFO:teuthology.task.install:The correct ceph version 18.0.0-1959.gb0d25925 is installed. 2023-01-25T05:00:44.403 DEBUG:teuthology.orchestra.run.smithi178:> sudo lsb_release -is 2023-01-25T05:00:44.455 INFO:teuthology.orchestra.run.smithi178.stdout:RedHatEnterprise 2023-01-25T05:00:44.456 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T05:00:44.457 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T05:00:44.607 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T05:00:44.631 INFO:teuthology.orchestra.run.smithi178.stdout:18.0.0-1959.gb0d25925.el8 2023-01-25T05:00:44.631 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1959.gb0d25925.el8 2023-01-25T05:00:44.631 INFO:teuthology.task.install:The correct ceph version 18.0.0-1959.gb0d25925 is installed. 2023-01-25T05:00:44.632 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-25T05:00:44.632 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:44.632 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T05:00:44.659 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:44.659 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T05:00:44.698 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-25T05:00:44.698 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:44.699 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T05:00:44.724 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T05:00:44.790 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:44.791 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T05:00:44.819 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T05:00:44.886 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-25T05:00:44.886 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:44.886 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T05:00:44.913 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T05:00:44.979 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:44.979 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T05:00:45.006 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T05:00:45.073 INFO:teuthology.run_tasks:Running task ceph... 2023-01-25T05:00:45.139 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-25T05:00:45.140 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /var/log/ceph 2023-01-25T05:00:45.142 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /var/log/ceph 2023-01-25T05:00:45.168 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-25T05:00:45.169 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T05:00:45.209 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T05:00:45.234 INFO:tasks.ceph:Creating extra log directories... 2023-01-25T05:00:45.235 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T05:00:45.276 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T05:00:45.313 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-25T05:00:45.313 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd sloppy crc': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon election default strategy': 1, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-25T05:00:45.314 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136784', 'branch': 'wip-yuri-testing-2023-01-23-0926', 'description': 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7136784', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-01-23-0926'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd sloppy crc': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'osd objectstore': 'filestore', 'osd sloppy crc': True}}, 'filestore': True, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-yuri-testing-2023-01-23-0926', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri-testing-2023-01-23-0926', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'targets': {'smithi135.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi178.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd sloppy crc': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon election default strategy': 1, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'cluster': 'ceph'}}, {'exec': {'mon.a': ['while ! ceph balancer status ; do sleep 1 ; done', 'ceph balancer mode crush-compat', 'ceph balancer on']}}, {'thrashosds': {'chance_pgnum_grow': 0.25, 'chance_pgnum_shrink': 0.25, 'chance_pgpnum_fix': 0.25, 'chance_test_map_discontinuity': 2, 'timeout': 1800}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache readproxy', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000, 'pool_snaps': True, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_22:20:59', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-25T05:00:45.315 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T05:00:45.342 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T05:00:45.358 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T05:00:45.398 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T05:00:45.425 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:45.425 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T05:00:45.479 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T05:00:45.480 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2023-01-25T05:00:45.536 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T05:00:45.536 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:00:45.537 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 337906 Links: 1 2023-01-25T05:00:45.537 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:00:45.537 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:00:45.537 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-01-25 05:00:31.901746644 +0000 2023-01-25T05:00:45.537 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-01-25 04:55:53.349606917 +0000 2023-01-25T05:00:45.537 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-01-25 04:55:53.349606917 +0000 2023-01-25T05:00:45.538 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-01-25T05:00:45.538 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T05:00:45.601 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-01-25T05:00:45.601 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-01-25T05:00:45.601 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000246949 s, 2.1 MB/s 2023-01-25T05:00:45.602 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T05:00:45.660 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2023-01-25T05:00:45.716 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T05:00:45.716 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:00:45.717 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 348957 Links: 1 2023-01-25T05:00:45.717 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:00:45.717 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:00:45.717 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-01-25 05:00:31.901746644 +0000 2023-01-25T05:00:45.717 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-01-25 04:55:53.863595889 +0000 2023-01-25T05:00:45.718 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-01-25 04:55:53.863595889 +0000 2023-01-25T05:00:45.718 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-01-25T05:00:45.718 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T05:00:45.781 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-01-25T05:00:45.781 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-01-25T05:00:45.781 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000241984 s, 2.1 MB/s 2023-01-25T05:00:45.782 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T05:00:45.839 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2023-01-25T05:00:45.896 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T05:00:45.896 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:00:45.896 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 351877 Links: 1 2023-01-25T05:00:45.897 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:00:45.897 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:00:45.897 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-01-25 05:00:31.902746623 +0000 2023-01-25T05:00:45.897 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-01-25 04:55:54.380584796 +0000 2023-01-25T05:00:45.898 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-01-25 04:55:54.380584796 +0000 2023-01-25T05:00:45.898 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-01-25T05:00:45.898 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T05:00:45.961 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-01-25T05:00:45.961 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-01-25T05:00:45.962 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000202511 s, 2.5 MB/s 2023-01-25T05:00:45.962 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T05:00:46.019 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2023-01-25T05:00:46.076 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T05:00:46.076 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:00:46.076 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 354075 Links: 1 2023-01-25T05:00:46.077 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:00:46.077 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:00:46.077 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-01-25 05:00:31.902746623 +0000 2023-01-25T05:00:46.077 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-01-25 04:55:54.874574197 +0000 2023-01-25T05:00:46.078 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-01-25 04:55:54.874574197 +0000 2023-01-25T05:00:46.078 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-01-25T05:00:46.078 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T05:00:46.141 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-01-25T05:00:46.141 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-01-25T05:00:46.141 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000192804 s, 2.7 MB/s 2023-01-25T05:00:46.142 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T05:00:46.200 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:00:46.200 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:46.200 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T05:00:46.221 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T05:00:46.221 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2023-01-25T05:00:46.281 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T05:00:46.281 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:00:46.281 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 354454 Links: 1 2023-01-25T05:00:46.282 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:00:46.282 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:00:46.282 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-01-25 05:00:24.587363195 +0000 2023-01-25T05:00:46.282 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-01-25 04:55:53.339607408 +0000 2023-01-25T05:00:46.282 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-01-25 04:55:53.339607408 +0000 2023-01-25T05:00:46.283 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-01-25T05:00:46.283 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T05:00:46.351 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-01-25T05:00:46.351 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-01-25T05:00:46.351 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000138493 s, 3.7 MB/s 2023-01-25T05:00:46.352 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T05:00:46.412 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2023-01-25T05:00:46.471 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T05:00:46.471 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:00:46.471 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 359428 Links: 1 2023-01-25T05:00:46.472 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:00:46.472 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:00:46.472 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-01-25 05:00:24.587363195 +0000 2023-01-25T05:00:46.472 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-01-25 04:55:53.838597823 +0000 2023-01-25T05:00:46.473 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-01-25 04:55:53.838597823 +0000 2023-01-25T05:00:46.473 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-01-25T05:00:46.473 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T05:00:46.538 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-01-25T05:00:46.539 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-01-25T05:00:46.539 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000137052 s, 3.7 MB/s 2023-01-25T05:00:46.540 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T05:00:46.599 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2023-01-25T05:00:46.657 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T05:00:46.658 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:00:46.658 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 346938 Links: 1 2023-01-25T05:00:46.658 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:00:46.658 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:00:46.658 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-01-25 05:00:24.587363195 +0000 2023-01-25T05:00:46.659 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-01-25 04:55:54.339588199 +0000 2023-01-25T05:00:46.659 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-01-25 04:55:54.339588199 +0000 2023-01-25T05:00:46.659 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-01-25T05:00:46.659 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T05:00:46.723 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-01-25T05:00:46.723 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-01-25T05:00:46.724 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000102472 s, 5.0 MB/s 2023-01-25T05:00:46.725 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T05:00:46.782 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2023-01-25T05:00:46.839 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T05:00:46.840 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T05:00:46.840 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 357358 Links: 1 2023-01-25T05:00:46.840 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T05:00:46.840 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T05:00:46.840 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-01-25 05:00:24.587363195 +0000 2023-01-25T05:00:46.841 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-01-25 04:55:54.827578825 +0000 2023-01-25T05:00:46.841 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-01-25 04:55:54.827578825 +0000 2023-01-25T05:00:46.841 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-01-25T05:00:46.841 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T05:00:46.904 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-01-25T05:00:46.905 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-01-25T05:00:46.905 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000116202 s, 4.4 MB/s 2023-01-25T05:00:46.906 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T05:00:46.964 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:00:46.964 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T05:00:46.965 INFO:tasks.ceph:Generating config... 2023-01-25T05:00:46.966 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-25T05:00:46.966 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-25T05:00:46.966 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-25T05:00:46.967 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-25T05:00:46.967 INFO:tasks.ceph:[osd] osd backoff on degraded = True 2023-01-25T05:00:46.967 INFO:tasks.ceph:[osd] osd backoff on peering = True 2023-01-25T05:00:46.968 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-01-25T05:00:46.968 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-25T05:00:46.968 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-25T05:00:46.968 INFO:tasks.ceph:[osd] osd inject bad map crc probability = 0.1 2023-01-25T05:00:46.969 INFO:tasks.ceph:[osd] osd map cache size = 1 2023-01-25T05:00:46.969 INFO:tasks.ceph:[osd] osd max backfills = 6 2023-01-25T05:00:46.969 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-01-25T05:00:46.969 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-25T05:00:46.970 INFO:tasks.ceph:[osd] osd objectstore = filestore 2023-01-25T05:00:46.970 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-25T05:00:46.970 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-25T05:00:46.970 INFO:tasks.ceph:[osd] osd scrub during recovery = False 2023-01-25T05:00:46.971 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-01-25T05:00:46.971 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-01-25T05:00:46.971 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-25T05:00:46.971 INFO:tasks.ceph:[osd] osd sloppy crc = True 2023-01-25T05:00:46.972 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-01-25T05:00:46.972 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-01-25T05:00:46.972 INFO:tasks.ceph:[global] mon election default strategy = 1 2023-01-25T05:00:46.972 INFO:tasks.ceph:[global] ms type = async 2023-01-25T05:00:46.973 INFO:tasks.ceph:[global] osd debug inject dispatch delay duration = 0.1 2023-01-25T05:00:46.973 INFO:tasks.ceph:[global] osd debug inject dispatch delay probability = 0.1 2023-01-25T05:00:46.973 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-01-25T05:00:46.973 INFO:tasks.ceph:[global] osd_pool_default_size = 2 2023-01-25T05:00:46.974 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-25T05:00:46.974 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-25T05:00:46.974 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-25T05:00:46.974 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-25T05:00:46.975 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-25T05:00:46.975 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-01-25T05:00:46.975 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-01-25T05:00:46.975 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-01-25T05:00:46.976 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-01-25T05:00:46.976 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-25T05:00:46.976 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-01-25T05:00:46.977 INFO:tasks.ceph:Setting up mon.a... 2023-01-25T05:00:46.977 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-25T05:00:47.031 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.keyring 2023-01-25T05:00:47.034 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-25T05:00:47.081 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T05:00:47.107 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.135'), ('mon.c', '[v2:172.21.15.135:3301,v1:172.21.15.135:6790]'), ('mon.b', '172.21.15.178')] 2023-01-25T05:00:47.108 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790],172.21.15.178', 'mon election default strategy': 1, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd sloppy crc': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-25T05:00:47.109 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:47.110 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T05:00:47.165 DEBUG:teuthology.orchestra.run.smithi135:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.135 --addv c '[v2:172.21.15.135:3301,v1:172.21.15.135:6790]' --add b 172.21.15.178 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:47.251 INFO:teuthology.orchestra.run.smithi135.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:47.252 INFO:teuthology.orchestra.run.smithi135.stdout:monmaptool: generated fsid 8729840b-5c6f-4675-8457-dfb384ce25b3 2023-01-25T05:00:47.252 INFO:teuthology.orchestra.run.smithi135.stdout:setting min_mon_release = pacific 2023-01-25T05:00:47.252 INFO:teuthology.orchestra.run.smithi135.stdout:epoch 0 2023-01-25T05:00:47.252 INFO:teuthology.orchestra.run.smithi135.stdout:fsid 8729840b-5c6f-4675-8457-dfb384ce25b3 2023-01-25T05:00:47.252 INFO:teuthology.orchestra.run.smithi135.stdout:last_changed 2023-01-25T05:00:47.251393+0000 2023-01-25T05:00:47.253 INFO:teuthology.orchestra.run.smithi135.stdout:created 2023-01-25T05:00:47.251393+0000 2023-01-25T05:00:47.253 INFO:teuthology.orchestra.run.smithi135.stdout:min_mon_release 16 (pacific) 2023-01-25T05:00:47.253 INFO:teuthology.orchestra.run.smithi135.stdout:election_strategy: 1 2023-01-25T05:00:47.253 INFO:teuthology.orchestra.run.smithi135.stdout:0: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.a 2023-01-25T05:00:47.253 INFO:teuthology.orchestra.run.smithi135.stdout:1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.b 2023-01-25T05:00:47.254 INFO:teuthology.orchestra.run.smithi135.stdout:2: [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] mon.c 2023-01-25T05:00:47.254 INFO:teuthology.orchestra.run.smithi135.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-25T05:00:47.255 DEBUG:teuthology.orchestra.run.smithi135:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T05:00:47.271 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8729840b-5c6f-4675-8457-dfb384ce25b3... 2023-01-25T05:00:47.273 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T05:00:47.313 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T05:00:47.356 INFO:teuthology.orchestra.run.smithi135.stdout:[global] 2023-01-25T05:00:47.356 INFO:teuthology.orchestra.run.smithi135.stdout: chdir = "" 2023-01-25T05:00:47.357 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T05:00:47.357 INFO:teuthology.orchestra.run.smithi135.stdout: auth supported = cephx 2023-01-25T05:00:47.357 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.357 INFO:teuthology.orchestra.run.smithi135.stdout: filestore xattr use omap = true 2023-01-25T05:00:47.357 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.358 INFO:teuthology.orchestra.run.smithi135.stdout: mon clock drift allowed = 1.000 2023-01-25T05:00:47.358 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.358 INFO:teuthology.orchestra.run.smithi135.stdout: osd crush chooseleaf type = 0 2023-01-25T05:00:47.358 INFO:teuthology.orchestra.run.smithi135.stdout: auth debug = true 2023-01-25T05:00:47.358 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.359 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on old message = true 2023-01-25T05:00:47.359 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on bug = true 2023-01-25T05:00:47.359 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.359 INFO:teuthology.orchestra.run.smithi135.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T05:00:47.359 INFO:teuthology.orchestra.run.smithi135.stdout: mon pg warn max object skew = 0 2023-01-25T05:00:47.359 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.360 INFO:teuthology.orchestra.run.smithi135.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T05:00:47.360 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T05:00:47.360 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.360 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default size = 2 2023-01-25T05:00:47.360 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.361 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow primary affinity = true 2023-01-25T05:00:47.361 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow pg remap = true 2023-01-25T05:00:47.361 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on legacy crush tunables = false 2023-01-25T05:00:47.361 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on crush straw calc version zero = false 2023-01-25T05:00:47.361 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on no sortbitwise = false 2023-01-25T05:00:47.362 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on osd down out interval zero = false 2023-01-25T05:00:47.362 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on too few osds = false 2023-01-25T05:00:47.362 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T05:00:47.362 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T05:00:47.362 INFO:teuthology.orchestra.run.smithi135.stdout: mon_allow_pool_size_one = true 2023-01-25T05:00:47.363 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.363 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T05:00:47.363 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.363 INFO:teuthology.orchestra.run.smithi135.stdout: osd default data pool replay window = 5 2023-01-25T05:00:47.363 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.364 INFO:teuthology.orchestra.run.smithi135.stdout: mon allow pool delete = true 2023-01-25T05:00:47.364 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.364 INFO:teuthology.orchestra.run.smithi135.stdout: mon cluster log file level = debug 2023-01-25T05:00:47.364 INFO:teuthology.orchestra.run.smithi135.stdout: debug asserts on shutdown = true 2023-01-25T05:00:47.364 INFO:teuthology.orchestra.run.smithi135.stdout: mon health detail to clog = false 2023-01-25T05:00:47.365 INFO:teuthology.orchestra.run.smithi135.stdout: mon host = "172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790],172.21.15.178" 2023-01-25T05:00:47.365 INFO:teuthology.orchestra.run.smithi135.stdout: mon election default strategy = 1 2023-01-25T05:00:47.365 INFO:teuthology.orchestra.run.smithi135.stdout: ms type = async 2023-01-25T05:00:47.365 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug inject dispatch delay duration = 0.1 2023-01-25T05:00:47.365 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug inject dispatch delay probability = 0.1 2023-01-25T05:00:47.366 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_min_size = 2 2023-01-25T05:00:47.366 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_size = 2 2023-01-25T05:00:47.366 INFO:teuthology.orchestra.run.smithi135.stdout: fsid = 8729840b-5c6f-4675-8457-dfb384ce25b3 2023-01-25T05:00:47.366 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.366 INFO:teuthology.orchestra.run.smithi135.stdout:[osd] 2023-01-25T05:00:47.366 INFO:teuthology.orchestra.run.smithi135.stdout: osd journal size = 100 2023-01-25T05:00:47.367 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.367 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub load threshold = 5.0 2023-01-25T05:00:47.367 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub max interval = 120 2023-01-25T05:00:47.367 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.367 INFO:teuthology.orchestra.run.smithi135.stdout: osd recover clone overlap = true 2023-01-25T05:00:47.368 INFO:teuthology.orchestra.run.smithi135.stdout: osd recovery max chunk = 1048576 2023-01-25T05:00:47.368 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.368 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug shutdown = true 2023-01-25T05:00:47.368 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug op order = true 2023-01-25T05:00:47.368 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify stray on activate = true 2023-01-25T05:00:47.369 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.369 INFO:teuthology.orchestra.run.smithi135.stdout: osd open classes on start = true 2023-01-25T05:00:47.369 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug pg log writeout = true 2023-01-25T05:00:47.369 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.369 INFO:teuthology.orchestra.run.smithi135.stdout: osd deep scrub update digest min age = 30 2023-01-25T05:00:47.370 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.370 INFO:teuthology.orchestra.run.smithi135.stdout: osd map max advance = 10 2023-01-25T05:00:47.370 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.370 INFO:teuthology.orchestra.run.smithi135.stdout: journal zero on create = true 2023-01-25T05:00:47.370 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.371 INFO:teuthology.orchestra.run.smithi135.stdout: filestore ondisk finisher threads = 3 2023-01-25T05:00:47.371 INFO:teuthology.orchestra.run.smithi135.stdout: filestore apply finisher threads = 3 2023-01-25T05:00:47.371 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.371 INFO:teuthology.orchestra.run.smithi135.stdout: bdev debug aio = true 2023-01-25T05:00:47.371 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug misdirected ops = true 2023-01-25T05:00:47.372 INFO:teuthology.orchestra.run.smithi135.stdout: debug monc = 20 2023-01-25T05:00:47.372 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore zero block detection = True 2023-01-25T05:00:47.372 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-01-25T05:00:47.372 INFO:teuthology.orchestra.run.smithi135.stdout: debug osd = 20 2023-01-25T05:00:47.372 INFO:teuthology.orchestra.run.smithi135.stdout: osd backoff on degraded = True 2023-01-25T05:00:47.373 INFO:teuthology.orchestra.run.smithi135.stdout: osd backoff on peering = True 2023-01-25T05:00:47.373 INFO:teuthology.orchestra.run.smithi135.stdout: osd blocked scrub grace period = 3600 2023-01-25T05:00:47.373 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify cached snaps = True 2023-01-25T05:00:47.373 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify missing on start = True 2023-01-25T05:00:47.373 INFO:teuthology.orchestra.run.smithi135.stdout: osd inject bad map crc probability = 0.1 2023-01-25T05:00:47.373 INFO:teuthology.orchestra.run.smithi135.stdout: osd map cache size = 1 2023-01-25T05:00:47.374 INFO:teuthology.orchestra.run.smithi135.stdout: osd max backfills = 6 2023-01-25T05:00:47.374 INFO:teuthology.orchestra.run.smithi135.stdout: osd max markdown count = 1000 2023-01-25T05:00:47.374 INFO:teuthology.orchestra.run.smithi135.stdout: osd mclock override recovery settings = True 2023-01-25T05:00:47.374 INFO:teuthology.orchestra.run.smithi135.stdout: osd objectstore = filestore 2023-01-25T05:00:47.374 INFO:teuthology.orchestra.run.smithi135.stdout: osd op queue = debug_random 2023-01-25T05:00:47.375 INFO:teuthology.orchestra.run.smithi135.stdout: osd op queue cut off = debug_random 2023-01-25T05:00:47.375 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub during recovery = False 2023-01-25T05:00:47.375 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub min interval = 60 2023-01-25T05:00:47.375 INFO:teuthology.orchestra.run.smithi135.stdout: osd shutdown pgref assert = True 2023-01-25T05:00:47.375 INFO:teuthology.orchestra.run.smithi135.stdout: osd sloppy crc = True 2023-01-25T05:00:47.376 INFO:teuthology.orchestra.run.smithi135.stdout: osd_max_pg_log_entries = 3000 2023-01-25T05:00:47.376 INFO:teuthology.orchestra.run.smithi135.stdout: osd_min_pg_log_entries = 3000 2023-01-25T05:00:47.376 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.376 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr] 2023-01-25T05:00:47.376 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-01-25T05:00:47.377 INFO:teuthology.orchestra.run.smithi135.stdout: debug mgr = 20 2023-01-25T05:00:47.377 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2023-01-25T05:00:47.377 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2023-01-25T05:00:47.377 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:00:47.377 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:00:47.377 INFO:teuthology.orchestra.run.smithi135.stdout: mgr/telemetry/nag = false 2023-01-25T05:00:47.378 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.378 INFO:teuthology.orchestra.run.smithi135.stdout:[mon] 2023-01-25T05:00:47.378 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-01-25T05:00:47.378 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2023-01-25T05:00:47.378 INFO:teuthology.orchestra.run.smithi135.stdout: debug paxos = 20 2023-01-25T05:00:47.379 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2023-01-25T05:00:47.379 INFO:teuthology.orchestra.run.smithi135.stdout: mon data avail warn = 5 2023-01-25T05:00:47.379 INFO:teuthology.orchestra.run.smithi135.stdout: mon mgr mkfs grace = 240 2023-01-25T05:00:47.379 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:00:47.379 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd reporter subtree level = osd 2023-01-25T05:00:47.380 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd prime pg temp = true 2023-01-25T05:00:47.380 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:00:47.380 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.380 INFO:teuthology.orchestra.run.smithi135.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T05:00:47.380 INFO:teuthology.orchestra.run.smithi135.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T05:00:47.381 INFO:teuthology.orchestra.run.smithi135.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T05:00:47.381 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.381 INFO:teuthology.orchestra.run.smithi135.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T05:00:47.381 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T05:00:47.381 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T05:00:47.382 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.382 INFO:teuthology.orchestra.run.smithi135.stdout: # 1m isn't quite enough 2023-01-25T05:00:47.382 INFO:teuthology.orchestra.run.smithi135.stdout: mon_down_mkfs_grace = 2m 2023-01-25T05:00:47.382 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.382 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_filestore_osds = false 2023-01-25T05:00:47.383 INFO:teuthology.orchestra.run.smithi135.stdout: mon min osdmap epochs = 50 2023-01-25T05:00:47.383 INFO:teuthology.orchestra.run.smithi135.stdout: mon osdmap full prune interval = 2 2023-01-25T05:00:47.383 INFO:teuthology.orchestra.run.smithi135.stdout: mon osdmap full prune min = 15 2023-01-25T05:00:47.383 INFO:teuthology.orchestra.run.smithi135.stdout: mon osdmap full prune txsize = 2 2023-01-25T05:00:47.383 INFO:teuthology.orchestra.run.smithi135.stdout: mon scrub interval = 300 2023-01-25T05:00:47.383 INFO:teuthology.orchestra.run.smithi135.stdout: paxos service trim min = 10 2023-01-25T05:00:47.384 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:47.384 INFO:teuthology.orchestra.run.smithi135.stdout:[client] 2023-01-25T05:00:47.384 INFO:teuthology.orchestra.run.smithi135.stdout: rgw cache enabled = true 2023-01-25T05:00:47.384 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable ops log = true 2023-01-25T05:00:47.384 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable usage log = true 2023-01-25T05:00:47.385 INFO:teuthology.orchestra.run.smithi135.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T05:00:47.385 INFO:teuthology.orchestra.run.smithi135.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T05:00:47.385 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.a] 2023-01-25T05:00:47.385 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.c] 2023-01-25T05:00:47.385 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.b] 2023-01-25T05:00:47.387 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2023-01-25T05:00:47.387 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2023-01-25T05:00:47.387 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T05:00:47.388 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2023-01-25T05:00:47.388 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.388 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2023-01-25T05:00:47.388 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.388 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2023-01-25T05:00:47.389 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.389 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2023-01-25T05:00:47.389 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2023-01-25T05:00:47.389 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.389 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2023-01-25T05:00:47.390 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2023-01-25T05:00:47.390 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.390 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T05:00:47.390 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2023-01-25T05:00:47.390 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.391 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T05:00:47.391 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T05:00:47.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.391 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2023-01-25T05:00:47.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.392 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2023-01-25T05:00:47.392 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2023-01-25T05:00:47.392 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2023-01-25T05:00:47.392 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2023-01-25T05:00:47.392 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2023-01-25T05:00:47.392 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2023-01-25T05:00:47.393 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2023-01-25T05:00:47.393 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T05:00:47.393 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T05:00:47.393 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2023-01-25T05:00:47.393 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.394 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T05:00:47.394 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.394 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2023-01-25T05:00:47.394 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.394 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2023-01-25T05:00:47.395 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.395 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2023-01-25T05:00:47.395 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2023-01-25T05:00:47.395 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2023-01-25T05:00:47.395 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790],172.21.15.178" 2023-01-25T05:00:47.396 INFO:teuthology.orchestra.run.smithi178.stdout: mon election default strategy = 1 2023-01-25T05:00:47.396 INFO:teuthology.orchestra.run.smithi178.stdout: ms type = async 2023-01-25T05:00:47.396 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug inject dispatch delay duration = 0.1 2023-01-25T05:00:47.396 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug inject dispatch delay probability = 0.1 2023-01-25T05:00:47.396 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_min_size = 2 2023-01-25T05:00:47.397 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_size = 2 2023-01-25T05:00:47.397 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = 8729840b-5c6f-4675-8457-dfb384ce25b3 2023-01-25T05:00:47.397 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.397 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2023-01-25T05:00:47.397 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2023-01-25T05:00:47.398 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.398 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2023-01-25T05:00:47.398 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 120 2023-01-25T05:00:47.398 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.398 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2023-01-25T05:00:47.398 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2023-01-25T05:00:47.399 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.399 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2023-01-25T05:00:47.399 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2023-01-25T05:00:47.399 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2023-01-25T05:00:47.399 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.400 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2023-01-25T05:00:47.400 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2023-01-25T05:00:47.400 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.400 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2023-01-25T05:00:47.400 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.401 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2023-01-25T05:00:47.401 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.401 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2023-01-25T05:00:47.401 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.401 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2023-01-25T05:00:47.401 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2023-01-25T05:00:47.402 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.402 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2023-01-25T05:00:47.402 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2023-01-25T05:00:47.402 INFO:teuthology.orchestra.run.smithi178.stdout: debug monc = 20 2023-01-25T05:00:47.402 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore zero block detection = True 2023-01-25T05:00:47.403 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2023-01-25T05:00:47.403 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2023-01-25T05:00:47.403 INFO:teuthology.orchestra.run.smithi178.stdout: osd backoff on degraded = True 2023-01-25T05:00:47.403 INFO:teuthology.orchestra.run.smithi178.stdout: osd backoff on peering = True 2023-01-25T05:00:47.403 INFO:teuthology.orchestra.run.smithi178.stdout: osd blocked scrub grace period = 3600 2023-01-25T05:00:47.404 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify cached snaps = True 2023-01-25T05:00:47.404 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify missing on start = True 2023-01-25T05:00:47.404 INFO:teuthology.orchestra.run.smithi178.stdout: osd inject bad map crc probability = 0.1 2023-01-25T05:00:47.404 INFO:teuthology.orchestra.run.smithi178.stdout: osd map cache size = 1 2023-01-25T05:00:47.404 INFO:teuthology.orchestra.run.smithi178.stdout: osd max backfills = 6 2023-01-25T05:00:47.405 INFO:teuthology.orchestra.run.smithi178.stdout: osd max markdown count = 1000 2023-01-25T05:00:47.405 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock override recovery settings = True 2023-01-25T05:00:47.405 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = filestore 2023-01-25T05:00:47.405 INFO:teuthology.orchestra.run.smithi178.stdout: osd op queue = debug_random 2023-01-25T05:00:47.405 INFO:teuthology.orchestra.run.smithi178.stdout: osd op queue cut off = debug_random 2023-01-25T05:00:47.406 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub during recovery = False 2023-01-25T05:00:47.406 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub min interval = 60 2023-01-25T05:00:47.406 INFO:teuthology.orchestra.run.smithi178.stdout: osd shutdown pgref assert = True 2023-01-25T05:00:47.406 INFO:teuthology.orchestra.run.smithi178.stdout: osd sloppy crc = True 2023-01-25T05:00:47.406 INFO:teuthology.orchestra.run.smithi178.stdout: osd_max_pg_log_entries = 3000 2023-01-25T05:00:47.406 INFO:teuthology.orchestra.run.smithi178.stdout: osd_min_pg_log_entries = 3000 2023-01-25T05:00:47.407 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.407 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2023-01-25T05:00:47.407 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2023-01-25T05:00:47.407 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2023-01-25T05:00:47.407 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2023-01-25T05:00:47.408 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2023-01-25T05:00:47.408 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:00:47.408 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:00:47.408 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2023-01-25T05:00:47.408 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.409 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2023-01-25T05:00:47.409 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2023-01-25T05:00:47.409 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2023-01-25T05:00:47.409 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2023-01-25T05:00:47.409 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2023-01-25T05:00:47.410 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2023-01-25T05:00:47.410 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2023-01-25T05:00:47.410 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2023-01-25T05:00:47.410 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2023-01-25T05:00:47.410 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2023-01-25T05:00:47.410 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2023-01-25T05:00:47.411 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.411 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T05:00:47.411 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T05:00:47.411 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T05:00:47.411 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.412 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T05:00:47.412 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T05:00:47.412 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T05:00:47.412 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.412 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2023-01-25T05:00:47.413 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2023-01-25T05:00:47.413 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.413 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2023-01-25T05:00:47.413 INFO:teuthology.orchestra.run.smithi178.stdout: mon min osdmap epochs = 50 2023-01-25T05:00:47.413 INFO:teuthology.orchestra.run.smithi178.stdout: mon osdmap full prune interval = 2 2023-01-25T05:00:47.414 INFO:teuthology.orchestra.run.smithi178.stdout: mon osdmap full prune min = 15 2023-01-25T05:00:47.414 INFO:teuthology.orchestra.run.smithi178.stdout: mon osdmap full prune txsize = 2 2023-01-25T05:00:47.414 INFO:teuthology.orchestra.run.smithi178.stdout: mon scrub interval = 300 2023-01-25T05:00:47.414 INFO:teuthology.orchestra.run.smithi178.stdout: paxos service trim min = 10 2023-01-25T05:00:47.414 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-01-25T05:00:47.415 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2023-01-25T05:00:47.415 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2023-01-25T05:00:47.415 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2023-01-25T05:00:47.415 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2023-01-25T05:00:47.415 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T05:00:47.416 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T05:00:47.416 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2023-01-25T05:00:47.416 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2023-01-25T05:00:47.416 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2023-01-25T05:00:47.416 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-25T05:00:47.417 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-25T05:00:47.483 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-25T05:00:47.484 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:47.484 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-25T05:00:47.500 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:47.501 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-25T05:00:47.556 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi135.front.sepia.ceph.com 2023-01-25T05:00:47.557 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:47.557 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T05:00:47.557 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T05:00:47.632 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:47.632 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:47.689 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi178.front.sepia.ceph.com 2023-01-25T05:00:47.689 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:47.690 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T05:00:47.690 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T05:00:47.728 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:47.728 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:47.785 INFO:tasks.ceph:Setting up mon nodes... 2023-01-25T05:00:47.785 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-25T05:00:47.786 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T05:00:47.838 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T05:00:47.841 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T05:00:47.916 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T05:00:47.918 INFO:tasks.ceph:Setting up mds nodes... 2023-01-25T05:00:47.919 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-25T05:00:47.919 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-25T05:00:47.963 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-25T05:00:47.975 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-25T05:00:48.019 INFO:teuthology.orchestra.run.smithi178.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-25T05:00:48.031 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-25T05:00:48.032 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T05:00:48.032 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-25T05:00:48.059 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:00:48.060 INFO:tasks.ceph:role: osd.0 2023-01-25T05:00:48.060 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com 2023-01-25T05:00:48.060 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T05:00:48.133 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:00:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:00:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:00:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:00:48.134 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:00:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:00:48.135 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:00:48.135 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:00:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:00:48.135 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:00:48.559 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2023-01-25T05:00:48.560 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2023-01-25T05:00:48.561 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-25T05:00:48.693 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-25T05:00:48.725 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-25T05:00:48.791 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:00:48.792 INFO:tasks.ceph:role: osd.1 2023-01-25T05:00:48.792 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com 2023-01-25T05:00:48.792 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T05:00:48.865 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:00:48.865 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:00:48.866 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:00:48.866 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:00:48.866 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:00:48.866 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:00:48.867 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:00:48.867 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:00:48.867 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:00:48.867 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:00:49.286 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2023-01-25T05:00:49.288 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2023-01-25T05:00:49.289 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-25T05:00:49.373 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-25T05:00:49.444 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-25T05:00:49.510 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:00:49.510 INFO:tasks.ceph:role: osd.2 2023-01-25T05:00:49.510 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com 2023-01-25T05:00:49.511 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T05:00:49.583 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:00:49.584 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:00:49.584 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:00:49.584 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:00:49.584 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:00:49.584 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:00:49.585 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:00:49.585 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:00:49.585 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:00:49.585 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:00:50.037 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2023-01-25T05:00:50.039 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2023-01-25T05:00:50.040 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-25T05:00:50.151 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-25T05:00:50.183 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-25T05:00:50.249 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T05:00:50.250 INFO:tasks.ceph:role: osd.3 2023-01-25T05:00:50.250 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com 2023-01-25T05:00:50.250 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T05:00:50.323 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:00:50.323 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:00:50.324 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:00:50.324 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:00:50.324 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:00:50.324 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:00:50.324 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:00:50.325 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:00:50.325 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:00:50.325 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:00:50.763 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2023-01-25T05:00:50.764 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2023-01-25T05:00:50.765 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-25T05:00:50.939 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-25T05:00:50.971 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:51.069 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.068+0000 7f49654db5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-25T05:00:51.073 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.073+0000 7f49654db5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-25T05:00:51.091 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.090+0000 7f49654db5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:51.091 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.090+0000 7f49654db5c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T05:00:51.328 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.327+0000 7f49654db5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:51.330 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.329+0000 7f49654db5c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:51.330 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.329+0000 7f49654db5c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:51.352 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-25T05:00:51.387 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:51.474 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.473+0000 7f2a1ad0b5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-25T05:00:51.474 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.474+0000 7f2a1ad0b5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-25T05:00:51.499 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.498+0000 7f2a1ad0b5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:51.499 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.498+0000 7f2a1ad0b5c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T05:00:51.645 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.644+0000 7f2a1ad0b5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:51.645 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.644+0000 7f2a1ad0b5c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:51.645 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.644+0000 7f2a1ad0b5c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:51.654 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-25T05:00:51.680 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:51.766 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.765+0000 7f8d517c15c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-25T05:00:51.766 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.765+0000 7f8d517c15c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-25T05:00:51.770 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.770+0000 7f8d517c15c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:51.771 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.770+0000 7f8d517c15c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T05:00:51.914 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.914+0000 7f8d517c15c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:51.917 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.916+0000 7f8d517c15c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:51.917 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:51.916+0000 7f8d517c15c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:51.927 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-25T05:00:51.954 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:52.039 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:52.038+0000 7f46b815e5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-25T05:00:52.039 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:52.039+0000 7f46b815e5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-25T05:00:52.046 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:52.046+0000 7f46b815e5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:52.046 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:52.046+0000 7f46b815e5c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T05:00:52.179 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:52.178+0000 7f46b815e5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:52.179 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:52.178+0000 7f46b815e5c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:52.179 INFO:teuthology.orchestra.run.smithi135.stderr:2023-01-25T05:00:52.178+0000 7f46b815e5c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:52.187 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-25T05:00:52.213 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-25T05:00:52.244 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:00:52.245 INFO:tasks.ceph:role: osd.4 2023-01-25T05:00:52.245 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com 2023-01-25T05:00:52.245 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T05:00:52.314 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:00:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:00:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:00:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:00:52.315 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:00:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:00:52.316 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:00:52.316 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:00:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:00:52.316 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:00:52.337 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2023-01-25T05:00:52.340 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2023-01-25T05:00:52.340 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-25T05:00:52.422 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-25T05:00:52.492 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-25T05:00:52.560 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:00:52.560 INFO:tasks.ceph:role: osd.5 2023-01-25T05:00:52.560 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com 2023-01-25T05:00:52.561 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T05:00:52.625 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:00:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:00:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:00:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:00:52.625 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:00:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:00:52.626 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:00:52.626 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:00:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:00:52.626 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:00:52.647 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2023-01-25T05:00:52.649 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2023-01-25T05:00:52.649 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-25T05:00:52.729 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-25T05:00:52.799 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-25T05:00:52.866 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:00:52.866 INFO:tasks.ceph:role: osd.6 2023-01-25T05:00:52.867 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com 2023-01-25T05:00:52.867 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T05:00:52.932 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:00:52.932 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:00:52.933 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:00:52.933 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:00:52.933 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:00:52.933 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:00:52.933 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:00:52.934 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:00:52.934 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:00:52.934 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:00:52.955 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2023-01-25T05:00:52.956 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2023-01-25T05:00:52.957 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-25T05:00:53.035 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-25T05:00:53.105 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-25T05:00:53.172 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T05:00:53.172 INFO:tasks.ceph:role: osd.7 2023-01-25T05:00:53.173 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com 2023-01-25T05:00:53.173 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T05:00:53.238 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T05:00:53.238 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T05:00:53.238 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T05:00:53.238 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T05:00:53.239 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T05:00:53.239 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2023-01-25T05:00:53.239 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T05:00:53.239 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T05:00:53.240 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T05:00:53.240 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T05:00:53.261 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2023-01-25T05:00:53.263 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2023-01-25T05:00:53.263 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-25T05:00:53.342 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-25T05:00:53.412 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:53.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.519+0000 7fef5e0fa5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-25T05:00:53.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.519+0000 7fef5e0fa5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-25T05:00:53.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.523+0000 7fef5e0fa5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:53.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.523+0000 7fef5e0fa5c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T05:00:53.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.579+0000 7fef5e0fa5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:53.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.579+0000 7fef5e0fa5c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:53.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.579+0000 7fef5e0fa5c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:53.588 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-25T05:00:53.623 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:53.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.708+0000 7f5af53875c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-25T05:00:53.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.708+0000 7f5af53875c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-25T05:00:53.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.712+0000 7f5af53875c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:53.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.712+0000 7f5af53875c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T05:00:53.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.769+0000 7f5af53875c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:53.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.769+0000 7f5af53875c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:53.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.769+0000 7f5af53875c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:53.778 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-25T05:00:53.804 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:53.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.888+0000 7f9130d965c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-25T05:00:53.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.888+0000 7f9130d965c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-25T05:00:53.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.892+0000 7f9130d965c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:53.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.892+0000 7f9130d965c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T05:00:53.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.948+0000 7f9130d965c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:53.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.948+0000 7f9130d965c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:53.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:53.948+0000 7f9130d965c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:53.957 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-25T05:00:53.983 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:54.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:54.068+0000 7f85a4aab5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-25T05:00:54.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:54.068+0000 7f85a4aab5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-25T05:00:54.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:54.072+0000 7f85a4aab5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:54.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:54.072+0000 7f85a4aab5c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T05:00:54.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:54.128+0000 7f85a4aab5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:54.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:54.128+0000 7f85a4aab5c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:54.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-01-25T05:00:54.128+0000 7f85a4aab5c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T05:00:54.137 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-25T05:00:54.163 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-25T05:00:54.164 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:54.164 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-25T05:00:54.196 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:54.196 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-25T05:00:54.266 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:54.267 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-25T05:00:54.335 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:54.335 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-25T05:00:54.403 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:54.404 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-25T05:00:54.470 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:54.470 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-25T05:00:54.497 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:54.497 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-25T05:00:54.564 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:54.564 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-25T05:00:54.630 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:54.630 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-25T05:00:54.696 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:54.697 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-25T05:00:54.763 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:54.763 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-25T05:00:54.781 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:54.782 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-25T05:00:54.820 INFO:tasks.ceph:Adding keys to all mons... 2023-01-25T05:00:54.820 DEBUG:teuthology.orchestra.run.smithi135:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T05:00:54.823 DEBUG:teuthology.orchestra.run.smithi178:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T05:00:54.862 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.y] 2023-01-25T05:00:54.862 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQB/t9BjCSr8MRAAu+G9yJathnU7DEzr9BmO5g== 2023-01-25T05:00:54.863 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.0] 2023-01-25T05:00:54.863 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCDt9BjNP0fBBAAdfu22njlm6nWs7uxbTxeOg== 2023-01-25T05:00:54.863 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.1] 2023-01-25T05:00:54.863 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCDt9BjcWpBHBAAQxswBtG93svaffwU6MFCAg== 2023-01-25T05:00:54.863 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.2] 2023-01-25T05:00:54.864 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCDt9BjKr+iLRAAiUgJvuK7odG0yNIsPoKkGQ== 2023-01-25T05:00:54.864 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.3] 2023-01-25T05:00:54.864 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCEt9Bj89VVAhAAmFhJAf7dVolcX1RZSkWx4g== 2023-01-25T05:00:54.864 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.x] 2023-01-25T05:00:54.864 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQB/t9Bj++SXNhAAfEoaeNRmRgCpMu+bk53oNA== 2023-01-25T05:00:54.865 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.4] 2023-01-25T05:00:54.865 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCFt9BjjH8IHxAA3JDigDMfygPYVqaXIy9rlw== 2023-01-25T05:00:54.865 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.5] 2023-01-25T05:00:54.865 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCFt9BjoP9LKhAAkvFRSSAYeaXRxs82wNW3JA== 2023-01-25T05:00:54.865 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.6] 2023-01-25T05:00:54.866 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCFt9BjpQkFNRAAM8EdTSHJfLGL7bDN+gJKAQ== 2023-01-25T05:00:54.866 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.7] 2023-01-25T05:00:54.866 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCGt9BjoyYkBBAAz3uOaQ9oNmMlJ0E+tpvBGA== 2023-01-25T05:00:54.866 INFO:teuthology.orchestra.run.smithi135.stdout:[client.0] 2023-01-25T05:00:54.866 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQB/t9Bjs5RhORAAclaktRWWTdckr5P/Z1HDyg== 2023-01-25T05:00:54.867 INFO:teuthology.orchestra.run.smithi135.stdout:[client.1] 2023-01-25T05:00:54.867 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQCAt9Bjc3geARAANX5CnhuUAI9ZoOX2MlmiaQ== 2023-01-25T05:00:54.884 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.y] 2023-01-25T05:00:54.884 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQB/t9BjCSr8MRAAu+G9yJathnU7DEzr9BmO5g== 2023-01-25T05:00:54.884 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.0] 2023-01-25T05:00:54.884 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQCDt9BjNP0fBBAAdfu22njlm6nWs7uxbTxeOg== 2023-01-25T05:00:54.885 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.1] 2023-01-25T05:00:54.885 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQCDt9BjcWpBHBAAQxswBtG93svaffwU6MFCAg== 2023-01-25T05:00:54.885 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.2] 2023-01-25T05:00:54.885 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQCDt9BjKr+iLRAAiUgJvuK7odG0yNIsPoKkGQ== 2023-01-25T05:00:54.885 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.3] 2023-01-25T05:00:54.886 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQCEt9Bj89VVAhAAmFhJAf7dVolcX1RZSkWx4g== 2023-01-25T05:00:54.886 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.x] 2023-01-25T05:00:54.886 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQB/t9Bj++SXNhAAfEoaeNRmRgCpMu+bk53oNA== 2023-01-25T05:00:54.886 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.4] 2023-01-25T05:00:54.886 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQCFt9BjjH8IHxAA3JDigDMfygPYVqaXIy9rlw== 2023-01-25T05:00:54.886 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.5] 2023-01-25T05:00:54.887 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQCFt9BjoP9LKhAAkvFRSSAYeaXRxs82wNW3JA== 2023-01-25T05:00:54.887 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.6] 2023-01-25T05:00:54.887 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQCFt9BjpQkFNRAAM8EdTSHJfLGL7bDN+gJKAQ== 2023-01-25T05:00:54.887 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.7] 2023-01-25T05:00:54.887 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQCGt9BjoyYkBBAAz3uOaQ9oNmMlJ0E+tpvBGA== 2023-01-25T05:00:54.888 INFO:teuthology.orchestra.run.smithi178.stdout:[client.0] 2023-01-25T05:00:54.888 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQB/t9Bjs5RhORAAclaktRWWTdckr5P/Z1HDyg== 2023-01-25T05:00:54.888 INFO:teuthology.orchestra.run.smithi178.stdout:[client.1] 2023-01-25T05:00:54.888 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQCAt9Bjc3geARAANX5CnhuUAI9ZoOX2MlmiaQ== 2023-01-25T05:00:54.889 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:00:54.911 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:00:54.981 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:54.984 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.036 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.039 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.092 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.095 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.148 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.151 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.203 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:00:55.205 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T05:00:55.258 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.261 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.313 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.315 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.368 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.371 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.423 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.426 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T05:00:55.478 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:00:55.481 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:00:55.536 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:00:55.538 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T05:00:55.594 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-25T05:00:55.594 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-25T05:00:55.622 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T05:00:55.719 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-25T05:00:55.746 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-25T05:00:55.812 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T05:00:55.910 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-25T05:00:55.936 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-25T05:00:55.964 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T05:00:56.063 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-25T05:00:56.089 DEBUG:teuthology.orchestra.run.smithi135:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:56.091 DEBUG:teuthology.orchestra.run.smithi178:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T05:00:56.145 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-25T05:00:56.145 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-25T05:00:56.146 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-25T05:00:56.148 INFO:tasks.ceph.mon.a:Started 2023-01-25T05:00:56.149 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-25T05:00:56.149 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-25T05:00:56.151 INFO:tasks.ceph.mon.c:Started 2023-01-25T05:00:56.151 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-25T05:00:56.152 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-25T05:00:56.187 INFO:tasks.ceph.mon.b:Started 2023-01-25T05:00:56.188 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-25T05:00:56.188 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-25T05:00:56.189 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-25T05:00:56.193 INFO:tasks.ceph.mgr.y:Started 2023-01-25T05:00:56.193 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-25T05:00:56.194 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-25T05:00:56.197 INFO:tasks.ceph.mgr.x:Started 2023-01-25T05:00:56.197 INFO:tasks.ceph:Setting crush tunables to default 2023-01-25T05:00:56.197 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd crush tunables default 2023-01-25T05:00:56.582 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:56.580+0000 7f6727b51200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T05:00:56.586 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:56.586+0000 7f0952598200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T05:00:56.613 INFO:teuthology.orchestra.run.smithi135.stderr:adjusted tunables profile to default 2023-01-25T05:00:56.624 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-25T05:00:56.624 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:56.625 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-25T05:00:56.646 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:56.645+0000 7f6727b51200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T05:00:56.650 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:56.651 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-25T05:00:56.654 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:56.654+0000 7f0952598200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T05:00:56.679 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:56.679 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-25T05:00:56.720 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:56.718+0000 7f6727b51200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T05:00:56.732 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:56.731+0000 7f0952598200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T05:00:56.748 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-01-25T05:00:56.748 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-25T05:00:56.797 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:56.795+0000 7f6727b51200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T05:00:56.815 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:56.815 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-25T05:00:56.816 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:56.815+0000 7f0952598200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T05:00:56.842 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:56.843 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-25T05:00:56.909 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:56.909 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-25T05:00:56.942 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:56.940+0000 7f6727b51200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T05:00:56.956 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:56.956+0000 7f0952598200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T05:00:56.968 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-01-25T05:00:56.968 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-25T05:00:57.034 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 721fadcb-cd61-46a6-a18b-73833eeff96f 0 2023-01-25T05:00:57.284 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:57.282+0000 7f6727b51200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T05:00:57.284 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:57.283+0000 7f0952598200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T05:00:57.287 INFO:teuthology.orchestra.run.smithi178.stdout:0 2023-01-25T05:00:57.300 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new b6793ed6-09d7-436d-a063-aa3a97463a92 1 2023-01-25T05:00:57.357 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:57.357+0000 7f0952598200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T05:00:57.364 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:57.363+0000 7f6727b51200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T05:00:57.494 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:57.493+0000 7f0952598200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T05:00:57.502 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:57.500+0000 7f6727b51200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T05:00:57.515 INFO:teuthology.orchestra.run.smithi178.stdout:1 2023-01-25T05:00:57.528 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 5b30b63e-a29f-4ab5-813e-453ced6184dc 2 2023-01-25T05:00:57.574 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:57.573+0000 7f0952598200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T05:00:57.587 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:57.585+0000 7f6727b51200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T05:00:57.750 INFO:teuthology.orchestra.run.smithi178.stdout:2 2023-01-25T05:00:57.767 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new e64ff366-6511-49aa-9562-32da645862e6 3 2023-01-25T05:00:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:3 2023-01-25T05:00:57.992 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 2a56593a-39d8-4bac-b749-902fd6a2f842 4 2023-01-25T05:00:58.005 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:58.005+0000 7f0952598200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T05:00:58.116 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:58.114+0000 7f6727b51200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T05:00:58.206 INFO:teuthology.orchestra.run.smithi178.stdout:4 2023-01-25T05:00:58.219 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 40596a50-ff52-43ab-b737-cba69586855e 5 2023-01-25T05:00:58.432 INFO:teuthology.orchestra.run.smithi178.stdout:5 2023-01-25T05:00:58.445 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 44b2e2c2-d15c-4e13-90e5-7038a4ae907e 6 2023-01-25T05:00:58.550 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:58.549+0000 7f0952598200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T05:00:58.684 INFO:teuthology.orchestra.run.smithi178.stdout:6 2023-01-25T05:00:58.694 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:58.692+0000 7f6727b51200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T05:00:58.697 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new d864c502-0546-4daa-b8b0-79228e0e61a5 7 2023-01-25T05:00:58.707 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:58.706+0000 7f0952598200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T05:00:58.858 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:58.857+0000 7f6727b51200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T05:00:58.913 INFO:teuthology.orchestra.run.smithi178.stdout:7 2023-01-25T05:00:58.925 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-25T05:00:58.926 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-25T05:00:58.928 INFO:tasks.ceph.osd.0:Started 2023-01-25T05:00:58.928 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-25T05:00:58.929 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-25T05:00:58.931 INFO:tasks.ceph.osd.1:Started 2023-01-25T05:00:58.931 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-25T05:00:58.931 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-25T05:00:58.933 INFO:tasks.ceph.osd.2:Started 2023-01-25T05:00:58.934 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-25T05:00:58.934 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-25T05:00:58.936 INFO:tasks.ceph.osd.3:Started 2023-01-25T05:00:58.936 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-25T05:00:58.937 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-25T05:00:58.939 INFO:tasks.ceph.osd.4:Started 2023-01-25T05:00:58.939 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-25T05:00:58.940 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-25T05:00:58.942 INFO:tasks.ceph.osd.5:Started 2023-01-25T05:00:58.942 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-25T05:00:58.942 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-25T05:00:58.944 INFO:tasks.ceph.osd.6:Started 2023-01-25T05:00:58.945 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-25T05:00:58.945 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-25T05:00:58.947 INFO:tasks.ceph.osd.7:Started 2023-01-25T05:00:58.948 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T05:00:59.026 INFO:tasks.ceph.osd.3.smithi135.stderr:2023-01-25T05:00:59.023+0000 7fd5b88595c0 -1 Falling back to public interface 2023-01-25T05:00:59.031 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-01-25T05:00:59.029+0000 7f6cdb42e5c0 -1 Falling back to public interface 2023-01-25T05:00:59.037 INFO:tasks.ceph.osd.3.smithi135.stderr:2023-01-25T05:00:59.036+0000 7fd5b88595c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:59.037 INFO:tasks.ceph.osd.3.smithi135.stderr:2023-01-25T05:00:59.036+0000 7fd5b88595c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.037 INFO:tasks.ceph.osd.3.smithi135.stderr:2023-01-25T05:00:59.036+0000 7fd5b88595c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.038 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-01-25T05:00:59.036+0000 7fb376a6b5c0 -1 Falling back to public interface 2023-01-25T05:00:59.041 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-01-25T05:00:59.040+0000 7fed5df725c0 -1 Falling back to public interface 2023-01-25T05:00:59.044 INFO:tasks.ceph.osd.1.smithi135.stderr:2023-01-25T05:00:59.043+0000 7f27728d75c0 -1 Falling back to public interface 2023-01-25T05:00:59.044 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-01-25T05:00:59.042+0000 7f6cdb42e5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:59.045 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-01-25T05:00:59.042+0000 7f6cdb42e5c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.045 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-01-25T05:00:59.042+0000 7f6cdb42e5c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.045 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-01-25T05:00:59.043+0000 7f8e63b5a5c0 -1 Falling back to public interface 2023-01-25T05:00:59.046 INFO:tasks.ceph.osd.0.smithi135.stderr:2023-01-25T05:00:59.044+0000 7f51dffd45c0 -1 Falling back to public interface 2023-01-25T05:00:59.047 INFO:tasks.ceph.osd.3.smithi135.stderr:2023-01-25T05:00:59.045+0000 7fd5b88595c0 -1 osd.3 0 log_to_monitors true 2023-01-25T05:00:59.047 INFO:tasks.ceph.osd.2.smithi135.stderr:2023-01-25T05:00:59.045+0000 7f327a95d5c0 -1 Falling back to public interface 2023-01-25T05:00:59.051 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-01-25T05:00:59.049+0000 7f6cdb42e5c0 -1 osd.7 0 log_to_monitors true 2023-01-25T05:00:59.057 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-01-25T05:00:59.056+0000 7fb376a6b5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:59.058 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-01-25T05:00:59.056+0000 7fb376a6b5c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.058 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-01-25T05:00:59.056+0000 7fb376a6b5c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.061 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-01-25T05:00:59.059+0000 7f8e63b5a5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:59.061 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-01-25T05:00:59.059+0000 7fed5df725c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:59.061 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-01-25T05:00:59.059+0000 7fed5df725c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.062 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-01-25T05:00:59.059+0000 7fed5df725c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.062 INFO:tasks.ceph.osd.1.smithi135.stderr:2023-01-25T05:00:59.061+0000 7f27728d75c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:59.062 INFO:tasks.ceph.osd.1.smithi135.stderr:2023-01-25T05:00:59.061+0000 7f27728d75c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.063 INFO:tasks.ceph.osd.1.smithi135.stderr:2023-01-25T05:00:59.061+0000 7f27728d75c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.064 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-01-25T05:00:59.059+0000 7f8e63b5a5c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.064 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-01-25T05:00:59.059+0000 7f8e63b5a5c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.065 INFO:tasks.ceph.osd.0.smithi135.stderr:2023-01-25T05:00:59.064+0000 7f51dffd45c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:59.065 INFO:tasks.ceph.osd.0.smithi135.stderr:2023-01-25T05:00:59.064+0000 7f51dffd45c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.065 INFO:tasks.ceph.osd.0.smithi135.stderr:2023-01-25T05:00:59.064+0000 7f51dffd45c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.066 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-01-25T05:00:59.064+0000 7fb376a6b5c0 -1 osd.5 0 log_to_monitors true 2023-01-25T05:00:59.068 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-01-25T05:00:59.066+0000 7f8e63b5a5c0 -1 osd.6 0 log_to_monitors true 2023-01-25T05:00:59.069 INFO:tasks.ceph.osd.1.smithi135.stderr:2023-01-25T05:00:59.069+0000 7f27728d75c0 -1 osd.1 0 log_to_monitors true 2023-01-25T05:00:59.069 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-01-25T05:00:59.068+0000 7fed5df725c0 -1 osd.4 0 log_to_monitors true 2023-01-25T05:00:59.072 INFO:tasks.ceph.osd.2.smithi135.stderr:2023-01-25T05:00:59.071+0000 7f327a95d5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T05:00:59.073 INFO:tasks.ceph.osd.2.smithi135.stderr:2023-01-25T05:00:59.072+0000 7f327a95d5c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.073 INFO:tasks.ceph.osd.2.smithi135.stderr:2023-01-25T05:00:59.072+0000 7f327a95d5c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T05:00:59.073 INFO:tasks.ceph.osd.0.smithi135.stderr:2023-01-25T05:00:59.072+0000 7f51dffd45c0 -1 osd.0 0 log_to_monitors true 2023-01-25T05:00:59.079 INFO:tasks.ceph.osd.2.smithi135.stderr:2023-01-25T05:00:59.079+0000 7f327a95d5c0 -1 osd.2 0 log_to_monitors true 2023-01-25T05:00:59.190 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:00:59.190 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":10,"fsid":"8729840b-5c6f-4675-8457-dfb384ce25b3","created":"2023-01-25T05:00:56.460333+0000","modified":"2023-01-25T05:00:58.909937+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T05:00:58.909937+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"721fadcb-cd61-46a6-a18b-73833eeff96f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b6793ed6-09d7-436d-a063-aa3a97463a92","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5b30b63e-a29f-4ab5-813e-453ced6184dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e64ff366-6511-49aa-9562-32da645862e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2a56593a-39d8-4bac-b749-902fd6a2f842","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"40596a50-ff52-43ab-b737-cba69586855e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"44b2e2c2-d15c-4e13-90e5-7038a4ae907e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d864c502-0546-4daa-b8b0-79228e0e61a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:00:59.207 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-25T05:00:59.207 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-25T05:00:59.442 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:59.441+0000 7f0952598200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:00:59.519 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:59.518+0000 7f0952598200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T05:00:59.569 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:59.568+0000 7f6727b51200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:00:59.644 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:59.643+0000 7f6727b51200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T05:00:59.654 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:59.654+0000 7f0952598200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:00:59.781 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:59.780+0000 7f6727b51200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T05:00:59.808 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:59.808+0000 7f0952598200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T05:00:59.938 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:00:59.936+0000 7f6727b51200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T05:00:59.939 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:00:59.939+0000 7f0952598200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T05:01:00.012 DEBUG:teuthology.orchestra.run.smithi135:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T05:01:00.068 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:01:00.066+0000 7f6727b51200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T05:01:00.216 INFO:teuthology.misc.health.smithi135.stdout: 2023-01-25T05:01:00.216 INFO:teuthology.misc.health.smithi135.stdout:{"epoch":11,"fsid":"8729840b-5c6f-4675-8457-dfb384ce25b3","created":"2023-01-25T05:00:56.460333+0000","modified":"2023-01-25T05:00:59.910930+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T05:00:58.909937+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"721fadcb-cd61-46a6-a18b-73833eeff96f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b6793ed6-09d7-436d-a063-aa3a97463a92","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5b30b63e-a29f-4ab5-813e-453ced6184dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e64ff366-6511-49aa-9562-32da645862e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2a56593a-39d8-4bac-b749-902fd6a2f842","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"40596a50-ff52-43ab-b737-cba69586855e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"44b2e2c2-d15c-4e13-90e5-7038a4ae907e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d864c502-0546-4daa-b8b0-79228e0e61a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:01:00.227 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-25T05:01:00.304 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:01:00.303+0000 7f0952598200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T05:01:00.421 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:01:00.419+0000 7f6727b51200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T05:01:00.453 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:01:00.452+0000 7f0952598200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T05:01:00.565 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:01:00.564+0000 7f0952598200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T05:01:00.571 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:01:00.569+0000 7f6727b51200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T05:01:00.683 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:01:00.681+0000 7f6727b51200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T05:01:00.708 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:01:00.707+0000 7f0952598200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T05:01:00.805 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:01:00.804+0000 7f0952598200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T05:01:00.827 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:01:00.826+0000 7f6727b51200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T05:01:00.915 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-01-25T05:01:00.914+0000 7fb36ae23700 -1 osd.5 0 waiting for initial osdmap 2023-01-25T05:01:00.916 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-01-25T05:01:00.914+0000 7f8e57f12700 -1 osd.6 0 waiting for initial osdmap 2023-01-25T05:01:00.916 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-01-25T05:01:00.914+0000 7fed5232a700 -1 osd.4 0 waiting for initial osdmap 2023-01-25T05:01:00.917 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-01-25T05:01:00.914+0000 7f6ccf7e6700 -1 osd.7 0 waiting for initial osdmap 2023-01-25T05:01:00.917 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-01-25T05:01:00.914+0000 7fb3599eb700 -1 osd.5 0 handle_osd_map injecting map crc failure 2023-01-25T05:01:00.917 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-01-25T05:01:00.914+0000 7f6cbe3ae700 -1 osd.7 0 handle_osd_map injecting map crc failure 2023-01-25T05:01:00.918 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-01-25T05:01:00.914+0000 7fed40ef2700 -1 osd.4 0 handle_osd_map injecting map crc failure 2023-01-25T05:01:00.918 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-01-25T05:01:00.914+0000 7f8e46ada700 -1 osd.6 0 handle_osd_map injecting map crc failure 2023-01-25T05:01:00.919 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-01-25T05:01:00.916+0000 7f8e46ada700 -1 osd.6 2 handle_osd_map injecting map crc failure 2023-01-25T05:01:00.919 INFO:tasks.ceph.osd.1.smithi135.stderr:2023-01-25T05:01:00.915+0000 7f2766c8f700 -1 osd.1 0 waiting for initial osdmap 2023-01-25T05:01:00.920 INFO:tasks.ceph.osd.2.smithi135.stderr:2023-01-25T05:01:00.915+0000 7f326ed15700 -1 osd.2 0 waiting for initial osdmap 2023-01-25T05:01:00.920 INFO:tasks.ceph.osd.1.smithi135.stderr:2023-01-25T05:01:00.915+0000 7f2755857700 -1 osd.1 0 handle_osd_map injecting map crc failure 2023-01-25T05:01:00.920 INFO:tasks.ceph.osd.0.smithi135.stderr:2023-01-25T05:01:00.916+0000 7f51d438c700 -1 osd.0 0 waiting for initial osdmap 2023-01-25T05:01:00.921 INFO:tasks.ceph.osd.3.smithi135.stderr:2023-01-25T05:01:00.916+0000 7fd5acc11700 -1 osd.3 0 waiting for initial osdmap 2023-01-25T05:01:00.921 INFO:tasks.ceph.osd.2.smithi135.stderr:2023-01-25T05:01:00.916+0000 7f325d8dd700 -1 osd.2 0 handle_osd_map injecting map crc failure 2023-01-25T05:01:00.922 INFO:tasks.ceph.osd.0.smithi135.stderr:2023-01-25T05:01:00.917+0000 7f51c2f54700 -1 osd.0 0 handle_osd_map injecting map crc failure 2023-01-25T05:01:00.922 INFO:tasks.ceph.osd.3.smithi135.stderr:2023-01-25T05:01:00.917+0000 7fd59b7d9700 -1 osd.3 0 handle_osd_map injecting map crc failure 2023-01-25T05:01:00.923 INFO:tasks.ceph.osd.2.smithi135.stderr:2023-01-25T05:01:00.918+0000 7f325d8dd700 -1 osd.2 2 handle_osd_map injecting map crc failure 2023-01-25T05:01:00.928 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:01:00.926+0000 7f6727b51200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T05:01:00.931 INFO:tasks.ceph.osd.2.smithi135.stderr:2023-01-25T05:01:00.931+0000 7f326a8f7700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:01:00.939 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-01-25T05:01:00.938+0000 7f8e53af4700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:01:01.026 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-01-25T05:01:01.025+0000 7fb366a05700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:01:01.027 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-01-25T05:01:01.025+0000 7f6ccb3c8700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:01:01.028 INFO:tasks.ceph.osd.1.smithi135.stderr:2023-01-25T05:01:01.028+0000 7f2762871700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:01:01.032 INFO:tasks.ceph.osd.0.smithi135.stderr:2023-01-25T05:01:01.031+0000 7f51cff6e700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:01:01.033 INFO:tasks.ceph.osd.3.smithi135.stderr:2023-01-25T05:01:01.032+0000 7fd5a87f3700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:01:01.036 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-01-25T05:01:01.035+0000 7fed4df0c700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T05:01:01.161 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:01:01.160+0000 7f0952598200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T05:01:01.291 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:01:01.289+0000 7f6727b51200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T05:01:01.737 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:01:01.736+0000 7f0952598200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T05:01:01.868 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-01-25T05:01:01.867+0000 7f6727b51200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T05:01:02.755 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:01:02.755+0000 7f090a952700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-25T05:01:02.756 INFO:tasks.ceph.mgr.y.smithi135.stderr:2023-01-25T05:01:02.755+0000 7f090a952700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-25T05:01:04.274 INFO:tasks.ceph.osd.1.smithi135.stderr:2023-01-25T05:01:04.274+0000 7f2753853700 -1 osd.1 14 handle_osd_map injecting map crc failure 2023-01-25T05:01:06.052 INFO:tasks.ceph.osd.0.smithi135.stderr:2023-01-25T05:01:06.051+0000 7f51c0f50700 -1 osd.0 16 handle_osd_map injecting map crc failure 2023-01-25T05:01:07.033 DEBUG:teuthology.orchestra.run.smithi135:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T05:01:07.352 INFO:teuthology.misc.health.smithi135.stdout: 2023-01-25T05:01:07.352 INFO:teuthology.misc.health.smithi135.stdout:{"epoch":17,"fsid":"8729840b-5c6f-4675-8457-dfb384ce25b3","created":"2023-01-25T05:00:56.460333+0000","modified":"2023-01-25T05:01:05.950002+0000","last_up_change":"2023-01-25T05:01:01.920210+0000","last_in_change":"2023-01-25T05:00:58.909937+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T05:01:03.787169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"721fadcb-cd61-46a6-a18b-73833eeff96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":88726},{"type":"v1","addr":"172.21.15.135:6814","nonce":88726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":88726},{"type":"v1","addr":"172.21.15.135:6819","nonce":88726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6823","nonce":88726},{"type":"v1","addr":"172.21.15.135:6824","nonce":88726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":88726},{"type":"v1","addr":"172.21.15.135:6822","nonce":88726}]},"public_addr":"172.21.15.135:6814/88726","cluster_addr":"172.21.15.135:6819/88726","heartbeat_back_addr":"172.21.15.135:6824/88726","heartbeat_front_addr":"172.21.15.135:6822/88726","state":["exists","up"]},{"osd":1,"uuid":"b6793ed6-09d7-436d-a063-aa3a97463a92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":88727},{"type":"v1","addr":"172.21.15.135:6809","nonce":88727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":88727},{"type":"v1","addr":"172.21.15.135:6811","nonce":88727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":88727},{"type":"v1","addr":"172.21.15.135:6820","nonce":88727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":88727},{"type":"v1","addr":"172.21.15.135:6815","nonce":88727}]},"public_addr":"172.21.15.135:6809/88727","cluster_addr":"172.21.15.135:6811/88727","heartbeat_back_addr":"172.21.15.135:6820/88727","heartbeat_front_addr":"172.21.15.135:6815/88727","state":["exists","up"]},{"osd":2,"uuid":"5b30b63e-a29f-4ab5-813e-453ced6184dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":88728},{"type":"v1","addr":"172.21.15.135:6825","nonce":88728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":88728},{"type":"v1","addr":"172.21.15.135:6827","nonce":88728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":88728},{"type":"v1","addr":"172.21.15.135:6831","nonce":88728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":88728},{"type":"v1","addr":"172.21.15.135:6829","nonce":88728}]},"public_addr":"172.21.15.135:6825/88728","cluster_addr":"172.21.15.135:6827/88728","heartbeat_back_addr":"172.21.15.135:6831/88728","heartbeat_front_addr":"172.21.15.135:6829/88728","state":["exists","up"]},{"osd":3,"uuid":"e64ff366-6511-49aa-9562-32da645862e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":88725},{"type":"v1","addr":"172.21.15.135:6801","nonce":88725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":88725},{"type":"v1","addr":"172.21.15.135:6803","nonce":88725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":88725},{"type":"v1","addr":"172.21.15.135:6807","nonce":88725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":88725},{"type":"v1","addr":"172.21.15.135:6805","nonce":88725}]},"public_addr":"172.21.15.135:6801/88725","cluster_addr":"172.21.15.135:6803/88725","heartbeat_back_addr":"172.21.15.135:6807/88725","heartbeat_front_addr":"172.21.15.135:6805/88725","state":["exists","up"]},{"osd":4,"uuid":"2a56593a-39d8-4bac-b749-902fd6a2f842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":88693},{"type":"v1","addr":"172.21.15.178:6816","nonce":88693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":88693},{"type":"v1","addr":"172.21.15.178:6820","nonce":88693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":88693},{"type":"v1","addr":"172.21.15.178:6828","nonce":88693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":88693},{"type":"v1","addr":"172.21.15.178:6824","nonce":88693}]},"public_addr":"172.21.15.178:6816/88693","cluster_addr":"172.21.15.178:6820/88693","heartbeat_back_addr":"172.21.15.178:6828/88693","heartbeat_front_addr":"172.21.15.178:6824/88693","state":["exists","up"]},{"osd":5,"uuid":"40596a50-ff52-43ab-b737-cba69586855e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":88692},{"type":"v1","addr":"172.21.15.178:6809","nonce":88692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":88692},{"type":"v1","addr":"172.21.15.178:6811","nonce":88692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6815","nonce":88692},{"type":"v1","addr":"172.21.15.178:6817","nonce":88692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":88692},{"type":"v1","addr":"172.21.15.178:6813","nonce":88692}]},"public_addr":"172.21.15.178:6809/88692","cluster_addr":"172.21.15.178:6811/88692","heartbeat_back_addr":"172.21.15.178:6817/88692","heartbeat_front_addr":"172.21.15.178:6813/88692","state":["exists","up"]},{"osd":6,"uuid":"44b2e2c2-d15c-4e13-90e5-7038a4ae907e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":88694},{"type":"v1","addr":"172.21.15.178:6821","nonce":88694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":88694},{"type":"v1","addr":"172.21.15.178:6825","nonce":88694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":88694},{"type":"v1","addr":"172.21.15.178:6831","nonce":88694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6827","nonce":88694},{"type":"v1","addr":"172.21.15.178:6829","nonce":88694}]},"public_addr":"172.21.15.178:6821/88694","cluster_addr":"172.21.15.178:6825/88694","heartbeat_back_addr":"172.21.15.178:6831/88694","heartbeat_front_addr":"172.21.15.178:6829/88694","state":["exists","up"]},{"osd":7,"uuid":"d864c502-0546-4daa-b8b0-79228e0e61a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":88691},{"type":"v1","addr":"172.21.15.178:6801","nonce":88691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":88691},{"type":"v1","addr":"172.21.15.178:6803","nonce":88691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":88691},{"type":"v1","addr":"172.21.15.178:6807","nonce":88691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":88691},{"type":"v1","addr":"172.21.15.178:6805","nonce":88691}]},"public_addr":"172.21.15.178:6801/88691","cluster_addr":"172.21.15.178:6803/88691","heartbeat_back_addr":"172.21.15.178:6807/88691","heartbeat_front_addr":"172.21.15.178:6805/88691","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.082259+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.046164+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.122635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.061615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.100045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.093255+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.103696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.006745+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:01:07.365 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-25T05:01:07.366 INFO:tasks.ceph:Creating RBD pool 2023-01-25T05:01:07.366 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-25T05:01:07.974 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'rbd' created 2023-01-25T05:01:07.986 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-25T05:01:08.969 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-01-25T05:01:08.968+0000 7f8e46ada700 -1 osd.6 18 handle_osd_map injecting map crc failure 2023-01-25T05:01:08.973 INFO:teuthology.orchestra.run.smithi135.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-25T05:01:08.986 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-25T05:01:08.987 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-25T05:01:08.987 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-25T05:01:08.988 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-25T05:01:08.988 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-25T05:01:09.360 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:01:09.372 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":4,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":88453},{"type":"v1","addr":"172.21.15.135:6833","nonce":88453}]},"active_addr":"172.21.15.135:6833/88453","active_change":"2023-01-25T05:01:01.737893+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4099,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":4109201144}]},{"addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":4281140897}]},{"addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":1312879938}]}]}} 2023-01-25T05:01:09.377 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-25T05:01:09.377 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-25T05:01:09.377 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T05:01:09.695 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:01:09.696 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":19,"fsid":"8729840b-5c6f-4675-8457-dfb384ce25b3","created":"2023-01-25T05:00:56.460333+0000","modified":"2023-01-25T05:01:08.963735+0000","last_up_change":"2023-01-25T05:01:01.920210+0000","last_in_change":"2023-01-25T05:00:58.909937+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T05:01:03.787169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T05:01:07.694574+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"721fadcb-cd61-46a6-a18b-73833eeff96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":88726},{"type":"v1","addr":"172.21.15.135:6814","nonce":88726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":88726},{"type":"v1","addr":"172.21.15.135:6819","nonce":88726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6823","nonce":88726},{"type":"v1","addr":"172.21.15.135:6824","nonce":88726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":88726},{"type":"v1","addr":"172.21.15.135:6822","nonce":88726}]},"public_addr":"172.21.15.135:6814/88726","cluster_addr":"172.21.15.135:6819/88726","heartbeat_back_addr":"172.21.15.135:6824/88726","heartbeat_front_addr":"172.21.15.135:6822/88726","state":["exists","up"]},{"osd":1,"uuid":"b6793ed6-09d7-436d-a063-aa3a97463a92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":88727},{"type":"v1","addr":"172.21.15.135:6809","nonce":88727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":88727},{"type":"v1","addr":"172.21.15.135:6811","nonce":88727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":88727},{"type":"v1","addr":"172.21.15.135:6820","nonce":88727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":88727},{"type":"v1","addr":"172.21.15.135:6815","nonce":88727}]},"public_addr":"172.21.15.135:6809/88727","cluster_addr":"172.21.15.135:6811/88727","heartbeat_back_addr":"172.21.15.135:6820/88727","heartbeat_front_addr":"172.21.15.135:6815/88727","state":["exists","up"]},{"osd":2,"uuid":"5b30b63e-a29f-4ab5-813e-453ced6184dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":88728},{"type":"v1","addr":"172.21.15.135:6825","nonce":88728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":88728},{"type":"v1","addr":"172.21.15.135:6827","nonce":88728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":88728},{"type":"v1","addr":"172.21.15.135:6831","nonce":88728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":88728},{"type":"v1","addr":"172.21.15.135:6829","nonce":88728}]},"public_addr":"172.21.15.135:6825/88728","cluster_addr":"172.21.15.135:6827/88728","heartbeat_back_addr":"172.21.15.135:6831/88728","heartbeat_front_addr":"172.21.15.135:6829/88728","state":["exists","up"]},{"osd":3,"uuid":"e64ff366-6511-49aa-9562-32da645862e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":88725},{"type":"v1","addr":"172.21.15.135:6801","nonce":88725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":88725},{"type":"v1","addr":"172.21.15.135:6803","nonce":88725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":88725},{"type":"v1","addr":"172.21.15.135:6807","nonce":88725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":88725},{"type":"v1","addr":"172.21.15.135:6805","nonce":88725}]},"public_addr":"172.21.15.135:6801/88725","cluster_addr":"172.21.15.135:6803/88725","heartbeat_back_addr":"172.21.15.135:6807/88725","heartbeat_front_addr":"172.21.15.135:6805/88725","state":["exists","up"]},{"osd":4,"uuid":"2a56593a-39d8-4bac-b749-902fd6a2f842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":88693},{"type":"v1","addr":"172.21.15.178:6816","nonce":88693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":88693},{"type":"v1","addr":"172.21.15.178:6820","nonce":88693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":88693},{"type":"v1","addr":"172.21.15.178:6828","nonce":88693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":88693},{"type":"v1","addr":"172.21.15.178:6824","nonce":88693}]},"public_addr":"172.21.15.178:6816/88693","cluster_addr":"172.21.15.178:6820/88693","heartbeat_back_addr":"172.21.15.178:6828/88693","heartbeat_front_addr":"172.21.15.178:6824/88693","state":["exists","up"]},{"osd":5,"uuid":"40596a50-ff52-43ab-b737-cba69586855e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":88692},{"type":"v1","addr":"172.21.15.178:6809","nonce":88692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":88692},{"type":"v1","addr":"172.21.15.178:6811","nonce":88692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6815","nonce":88692},{"type":"v1","addr":"172.21.15.178:6817","nonce":88692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":88692},{"type":"v1","addr":"172.21.15.178:6813","nonce":88692}]},"public_addr":"172.21.15.178:6809/88692","cluster_addr":"172.21.15.178:6811/88692","heartbeat_back_addr":"172.21.15.178:6817/88692","heartbeat_front_addr":"172.21.15.178:6813/88692","state":["exists","up"]},{"osd":6,"uuid":"44b2e2c2-d15c-4e13-90e5-7038a4ae907e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":88694},{"type":"v1","addr":"172.21.15.178:6821","nonce":88694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":88694},{"type":"v1","addr":"172.21.15.178:6825","nonce":88694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":88694},{"type":"v1","addr":"172.21.15.178:6831","nonce":88694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6827","nonce":88694},{"type":"v1","addr":"172.21.15.178:6829","nonce":88694}]},"public_addr":"172.21.15.178:6821/88694","cluster_addr":"172.21.15.178:6825/88694","heartbeat_back_addr":"172.21.15.178:6831/88694","heartbeat_front_addr":"172.21.15.178:6829/88694","state":["exists","up"]},{"osd":7,"uuid":"d864c502-0546-4daa-b8b0-79228e0e61a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":88691},{"type":"v1","addr":"172.21.15.178:6801","nonce":88691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":88691},{"type":"v1","addr":"172.21.15.178:6803","nonce":88691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":88691},{"type":"v1","addr":"172.21.15.178:6807","nonce":88691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":88691},{"type":"v1","addr":"172.21.15.178:6805","nonce":88691}]},"public_addr":"172.21.15.178:6801/88691","cluster_addr":"172.21.15.178:6803/88691","heartbeat_back_addr":"172.21.15.178:6807/88691","heartbeat_front_addr":"172.21.15.178:6805/88691","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.082259+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.046164+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.122635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.061615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.100045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.093255+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.103696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.006745+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:01:09.710 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-25T05:01:09.711 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T05:01:10.029 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:01:10.029 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":20,"fsid":"8729840b-5c6f-4675-8457-dfb384ce25b3","created":"2023-01-25T05:00:56.460333+0000","modified":"2023-01-25T05:01:09.977465+0000","last_up_change":"2023-01-25T05:01:01.920210+0000","last_in_change":"2023-01-25T05:00:58.909937+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T05:01:03.787169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T05:01:07.694574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"721fadcb-cd61-46a6-a18b-73833eeff96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":88726},{"type":"v1","addr":"172.21.15.135:6814","nonce":88726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":88726},{"type":"v1","addr":"172.21.15.135:6819","nonce":88726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6823","nonce":88726},{"type":"v1","addr":"172.21.15.135:6824","nonce":88726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":88726},{"type":"v1","addr":"172.21.15.135:6822","nonce":88726}]},"public_addr":"172.21.15.135:6814/88726","cluster_addr":"172.21.15.135:6819/88726","heartbeat_back_addr":"172.21.15.135:6824/88726","heartbeat_front_addr":"172.21.15.135:6822/88726","state":["exists","up"]},{"osd":1,"uuid":"b6793ed6-09d7-436d-a063-aa3a97463a92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":88727},{"type":"v1","addr":"172.21.15.135:6809","nonce":88727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":88727},{"type":"v1","addr":"172.21.15.135:6811","nonce":88727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":88727},{"type":"v1","addr":"172.21.15.135:6820","nonce":88727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":88727},{"type":"v1","addr":"172.21.15.135:6815","nonce":88727}]},"public_addr":"172.21.15.135:6809/88727","cluster_addr":"172.21.15.135:6811/88727","heartbeat_back_addr":"172.21.15.135:6820/88727","heartbeat_front_addr":"172.21.15.135:6815/88727","state":["exists","up"]},{"osd":2,"uuid":"5b30b63e-a29f-4ab5-813e-453ced6184dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":88728},{"type":"v1","addr":"172.21.15.135:6825","nonce":88728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":88728},{"type":"v1","addr":"172.21.15.135:6827","nonce":88728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":88728},{"type":"v1","addr":"172.21.15.135:6831","nonce":88728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":88728},{"type":"v1","addr":"172.21.15.135:6829","nonce":88728}]},"public_addr":"172.21.15.135:6825/88728","cluster_addr":"172.21.15.135:6827/88728","heartbeat_back_addr":"172.21.15.135:6831/88728","heartbeat_front_addr":"172.21.15.135:6829/88728","state":["exists","up"]},{"osd":3,"uuid":"e64ff366-6511-49aa-9562-32da645862e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":88725},{"type":"v1","addr":"172.21.15.135:6801","nonce":88725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":88725},{"type":"v1","addr":"172.21.15.135:6803","nonce":88725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":88725},{"type":"v1","addr":"172.21.15.135:6807","nonce":88725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":88725},{"type":"v1","addr":"172.21.15.135:6805","nonce":88725}]},"public_addr":"172.21.15.135:6801/88725","cluster_addr":"172.21.15.135:6803/88725","heartbeat_back_addr":"172.21.15.135:6807/88725","heartbeat_front_addr":"172.21.15.135:6805/88725","state":["exists","up"]},{"osd":4,"uuid":"2a56593a-39d8-4bac-b749-902fd6a2f842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":88693},{"type":"v1","addr":"172.21.15.178:6816","nonce":88693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":88693},{"type":"v1","addr":"172.21.15.178:6820","nonce":88693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":88693},{"type":"v1","addr":"172.21.15.178:6828","nonce":88693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":88693},{"type":"v1","addr":"172.21.15.178:6824","nonce":88693}]},"public_addr":"172.21.15.178:6816/88693","cluster_addr":"172.21.15.178:6820/88693","heartbeat_back_addr":"172.21.15.178:6828/88693","heartbeat_front_addr":"172.21.15.178:6824/88693","state":["exists","up"]},{"osd":5,"uuid":"40596a50-ff52-43ab-b737-cba69586855e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":88692},{"type":"v1","addr":"172.21.15.178:6809","nonce":88692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":88692},{"type":"v1","addr":"172.21.15.178:6811","nonce":88692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6815","nonce":88692},{"type":"v1","addr":"172.21.15.178:6817","nonce":88692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":88692},{"type":"v1","addr":"172.21.15.178:6813","nonce":88692}]},"public_addr":"172.21.15.178:6809/88692","cluster_addr":"172.21.15.178:6811/88692","heartbeat_back_addr":"172.21.15.178:6817/88692","heartbeat_front_addr":"172.21.15.178:6813/88692","state":["exists","up"]},{"osd":6,"uuid":"44b2e2c2-d15c-4e13-90e5-7038a4ae907e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":88694},{"type":"v1","addr":"172.21.15.178:6821","nonce":88694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":88694},{"type":"v1","addr":"172.21.15.178:6825","nonce":88694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":88694},{"type":"v1","addr":"172.21.15.178:6831","nonce":88694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6827","nonce":88694},{"type":"v1","addr":"172.21.15.178:6829","nonce":88694}]},"public_addr":"172.21.15.178:6821/88694","cluster_addr":"172.21.15.178:6825/88694","heartbeat_back_addr":"172.21.15.178:6831/88694","heartbeat_front_addr":"172.21.15.178:6829/88694","state":["exists","up"]},{"osd":7,"uuid":"d864c502-0546-4daa-b8b0-79228e0e61a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":88691},{"type":"v1","addr":"172.21.15.178:6801","nonce":88691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":88691},{"type":"v1","addr":"172.21.15.178:6803","nonce":88691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":88691},{"type":"v1","addr":"172.21.15.178:6807","nonce":88691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":88691},{"type":"v1","addr":"172.21.15.178:6805","nonce":88691}]},"public_addr":"172.21.15.178:6801/88691","cluster_addr":"172.21.15.178:6803/88691","heartbeat_back_addr":"172.21.15.178:6807/88691","heartbeat_front_addr":"172.21.15.178:6805/88691","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.082259+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.046164+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.122635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.061615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.100045+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.093255+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.103696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T05:01:00.006745+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T05:01:10.043 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-25T05:01:10.044 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-25T05:01:10.044 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-25T05:01:10.045 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-25T05:01:10.045 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-25T05:01:10.046 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-25T05:01:10.047 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-25T05:01:10.047 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-25T05:01:10.326 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:10.326 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T05:01:10.332 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:10.333 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T05:01:10.335 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:10.336 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T05:01:10.344 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:10.345 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T05:01:10.349 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:10.349 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T05:01:10.359 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:10.359 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T05:01:10.361 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:10.362 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T05:01:10.365 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:10.365 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T05:01:10.937 INFO:teuthology.orchestra.run.smithi135.stdout:55834574850 2023-01-25T05:01:10.968 INFO:teuthology.orchestra.run.smithi135.stdout:55834574850 2023-01-25T05:01:10.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2023-01-25T05:01:10.987 INFO:teuthology.orchestra.run.smithi135.stdout:55834574850 2023-01-25T05:01:10.987 INFO:teuthology.orchestra.run.smithi135.stdout:55834574850 2023-01-25T05:01:10.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2023-01-25T05:01:11.004 INFO:teuthology.orchestra.run.smithi135.stdout:55834574850 2023-01-25T05:01:11.008 INFO:teuthology.orchestra.run.smithi135.stdout:55834574850 2023-01-25T05:01:11.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2023-01-25T05:01:11.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2023-01-25T05:01:11.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2023-01-25T05:01:11.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2023-01-25T05:01:11.063 INFO:teuthology.orchestra.run.smithi135.stdout:55834574850 2023-01-25T05:01:11.074 INFO:teuthology.orchestra.run.smithi135.stdout:55834574850 2023-01-25T05:01:11.079 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2023-01-25T05:01:11.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2023-01-25T05:01:11.970 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T05:01:11.991 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T05:01:12.010 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T05:01:12.018 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T05:01:12.029 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T05:01:12.031 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T05:01:12.080 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T05:01:12.091 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T05:01:12.560 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:12.569 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:12.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2023-01-25T05:01:12.583 DEBUG:teuthology.parallel:result is None 2023-01-25T05:01:12.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2023-01-25T05:01:12.591 DEBUG:teuthology.parallel:result is None 2023-01-25T05:01:12.594 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:12.616 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2023-01-25T05:01:12.616 DEBUG:teuthology.parallel:result is None 2023-01-25T05:01:12.636 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:12.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2023-01-25T05:01:12.651 DEBUG:teuthology.parallel:result is None 2023-01-25T05:01:12.682 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:12.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2023-01-25T05:01:12.701 DEBUG:teuthology.parallel:result is None 2023-01-25T05:01:12.701 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:12.703 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:12.705 INFO:teuthology.orchestra.run.smithi135.stdout:55834574851 2023-01-25T05:01:12.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2023-01-25T05:01:12.721 DEBUG:teuthology.parallel:result is None 2023-01-25T05:01:12.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2023-01-25T05:01:12.722 DEBUG:teuthology.parallel:result is None 2023-01-25T05:01:12.724 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2023-01-25T05:01:12.724 DEBUG:teuthology.parallel:result is None 2023-01-25T05:01:12.725 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T05:01:12.725 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T05:01:13.081 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:01:13.083 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2023-01-25T05:01:13.095 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2023-01-25T05:01:11.747535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":524828,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":80,"num_write":112,"num_write_kb":1310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":6368260,"kb_used_data":743227448,"kb_used_omap":431,"kb_used_meta":0,"kb_avail":743226780,"statfs":{"total":767585320960,"available":761064222720,"internally_reserved":684032,"allocated":761064906752,"data_stored":761064906752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":442346,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":3,"commit_latency_ns":0,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196620,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":50,"num_write_kb":723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.777955"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.171224+0000","last_change":"2023-01-25T05:01:09.184555+0000","last_active":"2023-01-25T05:01:10.171224+0000","last_peered":"2023-01-25T05:01:10.171224+0000","last_clean":"2023-01-25T05:01:10.171224+0000","last_became_active":"2023-01-25T05:01:09.184223+0000","last_became_peered":"2023-01-25T05:01:09.184223+0000","last_unstale":"2023-01-25T05:01:10.171224+0000","last_undegraded":"2023-01-25T05:01:10.171224+0000","last_fullsized":"2023-01-25T05:01:10.171224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:09.234526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:09.986384+0000","last_change":"2023-01-25T05:01:09.171645+0000","last_active":"2023-01-25T05:01:09.986384+0000","last_peered":"2023-01-25T05:01:09.986384+0000","last_clean":"2023-01-25T05:01:09.986384+0000","last_became_active":"2023-01-25T05:01:09.171427+0000","last_became_peered":"2023-01-25T05:01:09.171427+0000","last_unstale":"2023-01-25T05:01:09.986384+0000","last_undegraded":"2023-01-25T05:01:09.986384+0000","last_fullsized":"2023-01-25T05:01:09.986384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:28.310165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.135115+0000","last_change":"2023-01-25T05:01:09.071326+0000","last_active":"2023-01-25T05:01:10.135115+0000","last_peered":"2023-01-25T05:01:10.135115+0000","last_clean":"2023-01-25T05:01:10.135115+0000","last_became_active":"2023-01-25T05:01:09.071043+0000","last_became_peered":"2023-01-25T05:01:09.071043+0000","last_unstale":"2023-01-25T05:01:10.135115+0000","last_undegraded":"2023-01-25T05:01:10.135115+0000","last_fullsized":"2023-01-25T05:01:10.135115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:28.945742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.228535+0000","last_change":"2023-01-25T05:01:08.972545+0000","last_active":"2023-01-25T05:01:10.228535+0000","last_peered":"2023-01-25T05:01:10.228535+0000","last_clean":"2023-01-25T05:01:10.228535+0000","last_became_active":"2023-01-25T05:01:08.972336+0000","last_became_peered":"2023-01-25T05:01:08.972336+0000","last_unstale":"2023-01-25T05:01:10.228535+0000","last_undegraded":"2023-01-25T05:01:10.228535+0000","last_fullsized":"2023-01-25T05:01:10.228535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:18.753500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'76","reported_seq":113,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.228851+0000","last_change":"2023-01-25T05:01:05.234766+0000","last_active":"2023-01-25T05:01:10.228851+0000","last_peered":"2023-01-25T05:01:10.228851+0000","last_clean":"2023-01-25T05:01:10.228851+0000","last_became_active":"2023-01-25T05:01:05.133831+0000","last_became_peered":"2023-01-25T05:01:05.133831+0000","last_unstale":"2023-01-25T05:01:10.228851+0000","last_undegraded":"2023-01-25T05:01:10.228851+0000","last_fullsized":"2023-01-25T05:01:10.228851+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:03.938898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:03.938898+0000","last_clean_scrub_stamp":"2023-01-25T05:01:03.938898+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:06.861570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524828,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":80,"num_write":112,"num_write_kb":1310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.188336+0000","last_change":"2023-01-25T05:01:09.070912+0000","last_active":"2023-01-25T05:01:10.188336+0000","last_peered":"2023-01-25T05:01:10.188336+0000","last_clean":"2023-01-25T05:01:10.188336+0000","last_became_active":"2023-01-25T05:01:09.070612+0000","last_became_peered":"2023-01-25T05:01:09.070612+0000","last_unstale":"2023-01-25T05:01:10.188336+0000","last_undegraded":"2023-01-25T05:01:10.188336+0000","last_fullsized":"2023-01-25T05:01:10.188336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:27.458590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.149001+0000","last_change":"2023-01-25T05:01:09.871074+0000","last_active":"2023-01-25T05:01:10.149001+0000","last_peered":"2023-01-25T05:01:10.149001+0000","last_clean":"2023-01-25T05:01:10.149001+0000","last_became_active":"2023-01-25T05:01:09.870775+0000","last_became_peered":"2023-01-25T05:01:09.870775+0000","last_unstale":"2023-01-25T05:01:10.149001+0000","last_undegraded":"2023-01-25T05:01:10.149001+0000","last_fullsized":"2023-01-25T05:01:10.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:27.378655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.188289+0000","last_change":"2023-01-25T05:01:09.071011+0000","last_active":"2023-01-25T05:01:10.188289+0000","last_peered":"2023-01-25T05:01:10.188289+0000","last_clean":"2023-01-25T05:01:10.188289+0000","last_became_active":"2023-01-25T05:01:09.070621+0000","last_became_peered":"2023-01-25T05:01:09.070621+0000","last_unstale":"2023-01-25T05:01:10.188289+0000","last_undegraded":"2023-01-25T05:01:10.188289+0000","last_fullsized":"2023-01-25T05:01:10.188289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:15.857186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:09.986418+0000","last_change":"2023-01-25T05:01:09.072711+0000","last_active":"2023-01-25T05:01:09.986418+0000","last_peered":"2023-01-25T05:01:09.986418+0000","last_clean":"2023-01-25T05:01:09.986418+0000","last_became_active":"2023-01-25T05:01:09.072473+0000","last_became_peered":"2023-01-25T05:01:09.072473+0000","last_unstale":"2023-01-25T05:01:09.986418+0000","last_undegraded":"2023-01-25T05:01:09.986418+0000","last_fullsized":"2023-01-25T05:01:09.986418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:26.900431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":524828,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":80,"num_write":112,"num_write_kb":1310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":796752,"kb_used_data":92902904,"kb_used_omap":92,"kb_used_meta":0,"kb_avail":92902628,"statfs":{"total":95948165120,"available":95132291072,"internally_reserved":282624,"allocated":95132573696,"data_stored":95132573696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":94579,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":81.957999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":80.096000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":84.015000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":80.364999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":73.072000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795796,"kb_used_data":92903612,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903584,"statfs":{"total":95948165120,"available":95133270016,"internally_reserved":28672,"allocated":95133298688,"data_stored":95133298688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43679,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.8699999999999992}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795860,"kb_used_data":92903556,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92903520,"statfs":{"total":95948165120,"available":95133204480,"internally_reserved":36864,"allocated":95133241344,"data_stored":95133241344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":54387,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":796740,"kb_used_data":92902908,"kb_used_omap":88,"kb_used_meta":0,"kb_avail":92902640,"statfs":{"total":95948165120,"available":95132303360,"internally_reserved":274432,"allocated":95132577792,"data_stored":95132577792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":90835,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.664000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795784,"kb_used_data":92903608,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903596,"statfs":{"total":95948165120,"available":95133282304,"internally_reserved":12288,"allocated":95133294592,"data_stored":95133294592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43679,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.038}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.4290000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795896,"kb_used_data":92903516,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":92903484,"statfs":{"total":95948165120,"available":95133167616,"internally_reserved":32768,"allocated":95133200384,"data_stored":95133200384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38417,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795644,"kb_used_data":92903736,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":92903736,"statfs":{"total":95948165120,"available":95133425664,"internally_reserved":0,"allocated":95133425664,"data_stored":95133425664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33047,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":31.378}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795788,"kb_used_data":92903608,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903592,"statfs":{"total":95948165120,"available":95133278208,"internally_reserved":16384,"allocated":95133294592,"data_stored":95133294592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43723,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]}],"pool_statfs":[]}} 2023-01-25T05:01:13.097 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T05:01:13.412 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:01:13.414 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2023-01-25T05:01:13.425 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2023-01-25T05:01:11.747535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":524828,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":80,"num_write":112,"num_write_kb":1310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":6368260,"kb_used_data":743227448,"kb_used_omap":431,"kb_used_meta":0,"kb_avail":743226780,"statfs":{"total":767585320960,"available":761064222720,"internally_reserved":684032,"allocated":761064906752,"data_stored":761064906752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":442346,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":3,"commit_latency_ns":0,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196620,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":50,"num_write_kb":723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.777955"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.171224+0000","last_change":"2023-01-25T05:01:09.184555+0000","last_active":"2023-01-25T05:01:10.171224+0000","last_peered":"2023-01-25T05:01:10.171224+0000","last_clean":"2023-01-25T05:01:10.171224+0000","last_became_active":"2023-01-25T05:01:09.184223+0000","last_became_peered":"2023-01-25T05:01:09.184223+0000","last_unstale":"2023-01-25T05:01:10.171224+0000","last_undegraded":"2023-01-25T05:01:10.171224+0000","last_fullsized":"2023-01-25T05:01:10.171224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:09.234526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:09.986384+0000","last_change":"2023-01-25T05:01:09.171645+0000","last_active":"2023-01-25T05:01:09.986384+0000","last_peered":"2023-01-25T05:01:09.986384+0000","last_clean":"2023-01-25T05:01:09.986384+0000","last_became_active":"2023-01-25T05:01:09.171427+0000","last_became_peered":"2023-01-25T05:01:09.171427+0000","last_unstale":"2023-01-25T05:01:09.986384+0000","last_undegraded":"2023-01-25T05:01:09.986384+0000","last_fullsized":"2023-01-25T05:01:09.986384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:28.310165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.135115+0000","last_change":"2023-01-25T05:01:09.071326+0000","last_active":"2023-01-25T05:01:10.135115+0000","last_peered":"2023-01-25T05:01:10.135115+0000","last_clean":"2023-01-25T05:01:10.135115+0000","last_became_active":"2023-01-25T05:01:09.071043+0000","last_became_peered":"2023-01-25T05:01:09.071043+0000","last_unstale":"2023-01-25T05:01:10.135115+0000","last_undegraded":"2023-01-25T05:01:10.135115+0000","last_fullsized":"2023-01-25T05:01:10.135115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:28.945742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.228535+0000","last_change":"2023-01-25T05:01:08.972545+0000","last_active":"2023-01-25T05:01:10.228535+0000","last_peered":"2023-01-25T05:01:10.228535+0000","last_clean":"2023-01-25T05:01:10.228535+0000","last_became_active":"2023-01-25T05:01:08.972336+0000","last_became_peered":"2023-01-25T05:01:08.972336+0000","last_unstale":"2023-01-25T05:01:10.228535+0000","last_undegraded":"2023-01-25T05:01:10.228535+0000","last_fullsized":"2023-01-25T05:01:10.228535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:18.753500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'76","reported_seq":113,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.228851+0000","last_change":"2023-01-25T05:01:05.234766+0000","last_active":"2023-01-25T05:01:10.228851+0000","last_peered":"2023-01-25T05:01:10.228851+0000","last_clean":"2023-01-25T05:01:10.228851+0000","last_became_active":"2023-01-25T05:01:05.133831+0000","last_became_peered":"2023-01-25T05:01:05.133831+0000","last_unstale":"2023-01-25T05:01:10.228851+0000","last_undegraded":"2023-01-25T05:01:10.228851+0000","last_fullsized":"2023-01-25T05:01:10.228851+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:03.938898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:03.938898+0000","last_clean_scrub_stamp":"2023-01-25T05:01:03.938898+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:06.861570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524828,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":80,"num_write":112,"num_write_kb":1310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.188336+0000","last_change":"2023-01-25T05:01:09.070912+0000","last_active":"2023-01-25T05:01:10.188336+0000","last_peered":"2023-01-25T05:01:10.188336+0000","last_clean":"2023-01-25T05:01:10.188336+0000","last_became_active":"2023-01-25T05:01:09.070612+0000","last_became_peered":"2023-01-25T05:01:09.070612+0000","last_unstale":"2023-01-25T05:01:10.188336+0000","last_undegraded":"2023-01-25T05:01:10.188336+0000","last_fullsized":"2023-01-25T05:01:10.188336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:27.458590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.149001+0000","last_change":"2023-01-25T05:01:09.871074+0000","last_active":"2023-01-25T05:01:10.149001+0000","last_peered":"2023-01-25T05:01:10.149001+0000","last_clean":"2023-01-25T05:01:10.149001+0000","last_became_active":"2023-01-25T05:01:09.870775+0000","last_became_peered":"2023-01-25T05:01:09.870775+0000","last_unstale":"2023-01-25T05:01:10.149001+0000","last_undegraded":"2023-01-25T05:01:10.149001+0000","last_fullsized":"2023-01-25T05:01:10.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:27.378655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.188289+0000","last_change":"2023-01-25T05:01:09.071011+0000","last_active":"2023-01-25T05:01:10.188289+0000","last_peered":"2023-01-25T05:01:10.188289+0000","last_clean":"2023-01-25T05:01:10.188289+0000","last_became_active":"2023-01-25T05:01:09.070621+0000","last_became_peered":"2023-01-25T05:01:09.070621+0000","last_unstale":"2023-01-25T05:01:10.188289+0000","last_undegraded":"2023-01-25T05:01:10.188289+0000","last_fullsized":"2023-01-25T05:01:10.188289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:15.857186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:09.986418+0000","last_change":"2023-01-25T05:01:09.072711+0000","last_active":"2023-01-25T05:01:09.986418+0000","last_peered":"2023-01-25T05:01:09.986418+0000","last_clean":"2023-01-25T05:01:09.986418+0000","last_became_active":"2023-01-25T05:01:09.072473+0000","last_became_peered":"2023-01-25T05:01:09.072473+0000","last_unstale":"2023-01-25T05:01:09.986418+0000","last_undegraded":"2023-01-25T05:01:09.986418+0000","last_fullsized":"2023-01-25T05:01:09.986418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:26.900431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":524828,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":80,"num_write":112,"num_write_kb":1310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":796752,"kb_used_data":92902904,"kb_used_omap":92,"kb_used_meta":0,"kb_avail":92902628,"statfs":{"total":95948165120,"available":95132291072,"internally_reserved":282624,"allocated":95132573696,"data_stored":95132573696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":94579,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":81.957999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":80.096000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":84.015000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":80.364999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":73.072000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795796,"kb_used_data":92903612,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903584,"statfs":{"total":95948165120,"available":95133270016,"internally_reserved":28672,"allocated":95133298688,"data_stored":95133298688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43679,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.8699999999999992}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795860,"kb_used_data":92903556,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92903520,"statfs":{"total":95948165120,"available":95133204480,"internally_reserved":36864,"allocated":95133241344,"data_stored":95133241344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":54387,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":796740,"kb_used_data":92902908,"kb_used_omap":88,"kb_used_meta":0,"kb_avail":92902640,"statfs":{"total":95948165120,"available":95132303360,"internally_reserved":274432,"allocated":95132577792,"data_stored":95132577792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":90835,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.664000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795784,"kb_used_data":92903608,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903596,"statfs":{"total":95948165120,"available":95133282304,"internally_reserved":12288,"allocated":95133294592,"data_stored":95133294592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43679,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.038}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.4290000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795896,"kb_used_data":92903516,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":92903484,"statfs":{"total":95948165120,"available":95133167616,"internally_reserved":32768,"allocated":95133200384,"data_stored":95133200384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38417,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795644,"kb_used_data":92903736,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":92903736,"statfs":{"total":95948165120,"available":95133425664,"internally_reserved":0,"allocated":95133425664,"data_stored":95133425664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33047,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":31.378}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795788,"kb_used_data":92903608,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903592,"statfs":{"total":95948165120,"available":95133278208,"internally_reserved":16384,"allocated":95133294592,"data_stored":95133294592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43723,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]}],"pool_statfs":[]}} 2023-01-25T05:01:13.427 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T05:01:13.427 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-25T05:01:13.427 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-25T05:01:13.428 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-25T05:01:13.786 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:01:13.786 INFO:teuthology.orchestra.run.smithi135.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-25T05:01:13.800 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-25T05:01:13.800 INFO:teuthology.run_tasks:Running task exec... 2023-01-25T05:01:13.810 INFO:teuthology.task.exec:Executing custom commands... 2023-01-25T05:01:13.810 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi135.front.sepia.ceph.com 2023-01-25T05:01:13.811 DEBUG:teuthology.orchestra.run.smithi135:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'while ! ceph balancer status ; do sleep 1 ; done' 2023-01-25T05:01:14.121 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2023-01-25T05:01:14.122 INFO:teuthology.orchestra.run.smithi135.stdout: "active": true, 2023-01-25T05:01:14.122 INFO:teuthology.orchestra.run.smithi135.stdout: "last_optimize_duration": "0:00:00.000310", 2023-01-25T05:01:14.122 INFO:teuthology.orchestra.run.smithi135.stdout: "last_optimize_started": "Wed Jan 25 05:01:01 2023", 2023-01-25T05:01:14.123 INFO:teuthology.orchestra.run.smithi135.stdout: "mode": "upmap", 2023-01-25T05:01:14.123 INFO:teuthology.orchestra.run.smithi135.stdout: "no_optimization_needed": false, 2023-01-25T05:01:14.123 INFO:teuthology.orchestra.run.smithi135.stdout: "optimize_result": "No pools available", 2023-01-25T05:01:14.123 INFO:teuthology.orchestra.run.smithi135.stdout: "plans": [] 2023-01-25T05:01:14.123 INFO:teuthology.orchestra.run.smithi135.stdout:} 2023-01-25T05:01:14.136 DEBUG:teuthology.orchestra.run.smithi135:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer mode crush-compat' 2023-01-25T05:01:15.293 DEBUG:teuthology.orchestra.run.smithi135:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer on' 2023-01-25T05:01:15.623 INFO:teuthology.run_tasks:Running task thrashosds... 2023-01-25T05:01:15.634 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 0.25, 'chance_pgnum_shrink': 0.25, 'chance_pgpnum_fix': 0.25, 'chance_test_map_discontinuity': 2, 'timeout': 1800, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-01-25T05:01:15.634 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri-testing-2023-01-23-0926'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'ms type': 'async', 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd sloppy crc': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'osd objectstore': 'filestore', 'osd sloppy crc': True}}, 'filestore': True, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-yuri-testing-2023-01-23-0926', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}} 2023-01-25T05:01:15.634 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 0.25, 'chance_pgnum_shrink': 0.25, 'chance_pgpnum_fix': 0.25, 'chance_test_map_discontinuity': 2, 'timeout': 1800, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-01-25T05:01:15.635 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-01-25T05:01:15.967 INFO:tasks.thrashosds:Beginning thrashosds... 2023-01-25T05:01:15.968 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T05:01:15.968 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T05:01:16.280 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:01:16.282 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2023-01-25T05:01:16.293 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-01-25T05:01:15.748255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":89,"num_write":143,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":6371412,"kb_used_data":743226072,"kb_used_omap":464,"kb_used_meta":0,"kb_avail":743223628,"statfs":{"total":767585320960,"available":761060995072,"internally_reserved":2502656,"allocated":761063497728,"data_stored":761063497728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":475980,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,0,0,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":262160,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":45,"num_write":81,"num_write_kb":1567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.773290"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.171224+0000","last_change":"2023-01-25T05:01:09.184555+0000","last_active":"2023-01-25T05:01:10.171224+0000","last_peered":"2023-01-25T05:01:10.171224+0000","last_clean":"2023-01-25T05:01:10.171224+0000","last_became_active":"2023-01-25T05:01:09.184223+0000","last_became_peered":"2023-01-25T05:01:09.184223+0000","last_unstale":"2023-01-25T05:01:10.171224+0000","last_undegraded":"2023-01-25T05:01:10.171224+0000","last_fullsized":"2023-01-25T05:01:10.171224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:09.234526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:09.986384+0000","last_change":"2023-01-25T05:01:09.171645+0000","last_active":"2023-01-25T05:01:09.986384+0000","last_peered":"2023-01-25T05:01:09.986384+0000","last_clean":"2023-01-25T05:01:09.986384+0000","last_became_active":"2023-01-25T05:01:09.171427+0000","last_became_peered":"2023-01-25T05:01:09.171427+0000","last_unstale":"2023-01-25T05:01:09.986384+0000","last_undegraded":"2023-01-25T05:01:09.986384+0000","last_fullsized":"2023-01-25T05:01:09.986384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:28.310165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.135115+0000","last_change":"2023-01-25T05:01:09.071326+0000","last_active":"2023-01-25T05:01:10.135115+0000","last_peered":"2023-01-25T05:01:10.135115+0000","last_clean":"2023-01-25T05:01:10.135115+0000","last_became_active":"2023-01-25T05:01:09.071043+0000","last_became_peered":"2023-01-25T05:01:09.071043+0000","last_unstale":"2023-01-25T05:01:10.135115+0000","last_undegraded":"2023-01-25T05:01:10.135115+0000","last_fullsized":"2023-01-25T05:01:10.135115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:28.945742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.228535+0000","last_change":"2023-01-25T05:01:08.972545+0000","last_active":"2023-01-25T05:01:10.228535+0000","last_peered":"2023-01-25T05:01:10.228535+0000","last_clean":"2023-01-25T05:01:10.228535+0000","last_became_active":"2023-01-25T05:01:08.972336+0000","last_became_peered":"2023-01-25T05:01:08.972336+0000","last_unstale":"2023-01-25T05:01:10.228535+0000","last_undegraded":"2023-01-25T05:01:10.228535+0000","last_fullsized":"2023-01-25T05:01:10.228535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:18.753500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'104","reported_seq":147,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:12.675893+0000","last_change":"2023-01-25T05:01:05.234766+0000","last_active":"2023-01-25T05:01:12.675893+0000","last_peered":"2023-01-25T05:01:12.675893+0000","last_clean":"2023-01-25T05:01:12.675893+0000","last_became_active":"2023-01-25T05:01:05.133831+0000","last_became_peered":"2023-01-25T05:01:05.133831+0000","last_unstale":"2023-01-25T05:01:12.675893+0000","last_undegraded":"2023-01-25T05:01:12.675893+0000","last_fullsized":"2023-01-25T05:01:12.675893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:03.938898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:03.938898+0000","last_clean_scrub_stamp":"2023-01-25T05:01:03.938898+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:06.861570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":89,"num_write":143,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.188336+0000","last_change":"2023-01-25T05:01:09.070912+0000","last_active":"2023-01-25T05:01:10.188336+0000","last_peered":"2023-01-25T05:01:10.188336+0000","last_clean":"2023-01-25T05:01:10.188336+0000","last_became_active":"2023-01-25T05:01:09.070612+0000","last_became_peered":"2023-01-25T05:01:09.070612+0000","last_unstale":"2023-01-25T05:01:10.188336+0000","last_undegraded":"2023-01-25T05:01:10.188336+0000","last_fullsized":"2023-01-25T05:01:10.188336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:27.458590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.149001+0000","last_change":"2023-01-25T05:01:09.871074+0000","last_active":"2023-01-25T05:01:10.149001+0000","last_peered":"2023-01-25T05:01:10.149001+0000","last_clean":"2023-01-25T05:01:10.149001+0000","last_became_active":"2023-01-25T05:01:09.870775+0000","last_became_peered":"2023-01-25T05:01:09.870775+0000","last_unstale":"2023-01-25T05:01:10.149001+0000","last_undegraded":"2023-01-25T05:01:10.149001+0000","last_fullsized":"2023-01-25T05:01:10.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:27.378655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.188289+0000","last_change":"2023-01-25T05:01:09.071011+0000","last_active":"2023-01-25T05:01:10.188289+0000","last_peered":"2023-01-25T05:01:10.188289+0000","last_clean":"2023-01-25T05:01:10.188289+0000","last_became_active":"2023-01-25T05:01:09.070621+0000","last_became_peered":"2023-01-25T05:01:09.070621+0000","last_unstale":"2023-01-25T05:01:10.188289+0000","last_undegraded":"2023-01-25T05:01:10.188289+0000","last_fullsized":"2023-01-25T05:01:10.188289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:15.857186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:09.986418+0000","last_change":"2023-01-25T05:01:09.072711+0000","last_active":"2023-01-25T05:01:09.986418+0000","last_peered":"2023-01-25T05:01:09.986418+0000","last_clean":"2023-01-25T05:01:09.986418+0000","last_became_active":"2023-01-25T05:01:09.072473+0000","last_became_peered":"2023-01-25T05:01:09.072473+0000","last_unstale":"2023-01-25T05:01:09.986418+0000","last_undegraded":"2023-01-25T05:01:09.986418+0000","last_fullsized":"2023-01-25T05:01:09.986418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:26.900431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":89,"num_write":143,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798208,"kb_used_data":92902320,"kb_used_omap":108,"kb_used_meta":0,"kb_avail":92901172,"statfs":{"total":95948165120,"available":95130800128,"internally_reserved":1175552,"allocated":95131975680,"data_stored":95131975680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":111320,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,0,0,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795856,"kb_used_data":92903560,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903524,"statfs":{"total":95948165120,"available":95133208576,"internally_reserved":36864,"allocated":95133245440,"data_stored":95133245440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43717,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":74.756}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795860,"kb_used_data":92903556,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92903520,"statfs":{"total":95948165120,"available":95133204480,"internally_reserved":36864,"allocated":95133241344,"data_stored":95133241344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":54387,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798196,"kb_used_data":92902324,"kb_used_omap":105,"kb_used_meta":0,"kb_avail":92901184,"statfs":{"total":95948165120,"available":95130812416,"internally_reserved":1167360,"allocated":95131979776,"data_stored":95131979776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":107576,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.664000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795844,"kb_used_data":92903556,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903536,"statfs":{"total":95948165120,"available":95133220864,"internally_reserved":20480,"allocated":95133241344,"data_stored":95133241344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43717,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.038}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.4290000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795896,"kb_used_data":92903516,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":92903484,"statfs":{"total":95948165120,"available":95133167616,"internally_reserved":32768,"allocated":95133200384,"data_stored":95133200384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38417,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.6289999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.282}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795704,"kb_used_data":92903684,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":92903676,"statfs":{"total":95948165120,"available":95133364224,"internally_reserved":8192,"allocated":95133372416,"data_stored":95133372416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33085,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":31.378}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795848,"kb_used_data":92903556,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903532,"statfs":{"total":95948165120,"available":95133216768,"internally_reserved":24576,"allocated":95133241344,"data_stored":95133241344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43761,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]}],"pool_statfs":[]}} 2023-01-25T05:01:16.295 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T05:01:16.603 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-01-25T05:01:16.606 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2023-01-25T05:01:16.617 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-01-25T05:01:15.748255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":89,"num_write":143,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":6371412,"kb_used_data":743226072,"kb_used_omap":464,"kb_used_meta":0,"kb_avail":743223628,"statfs":{"total":767585320960,"available":761060995072,"internally_reserved":2502656,"allocated":761063497728,"data_stored":761063497728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":475980,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,0,0,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":262160,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":45,"num_write":81,"num_write_kb":1567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.773290"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.171224+0000","last_change":"2023-01-25T05:01:09.184555+0000","last_active":"2023-01-25T05:01:10.171224+0000","last_peered":"2023-01-25T05:01:10.171224+0000","last_clean":"2023-01-25T05:01:10.171224+0000","last_became_active":"2023-01-25T05:01:09.184223+0000","last_became_peered":"2023-01-25T05:01:09.184223+0000","last_unstale":"2023-01-25T05:01:10.171224+0000","last_undegraded":"2023-01-25T05:01:10.171224+0000","last_fullsized":"2023-01-25T05:01:10.171224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:09.234526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:09.986384+0000","last_change":"2023-01-25T05:01:09.171645+0000","last_active":"2023-01-25T05:01:09.986384+0000","last_peered":"2023-01-25T05:01:09.986384+0000","last_clean":"2023-01-25T05:01:09.986384+0000","last_became_active":"2023-01-25T05:01:09.171427+0000","last_became_peered":"2023-01-25T05:01:09.171427+0000","last_unstale":"2023-01-25T05:01:09.986384+0000","last_undegraded":"2023-01-25T05:01:09.986384+0000","last_fullsized":"2023-01-25T05:01:09.986384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:28.310165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.135115+0000","last_change":"2023-01-25T05:01:09.071326+0000","last_active":"2023-01-25T05:01:10.135115+0000","last_peered":"2023-01-25T05:01:10.135115+0000","last_clean":"2023-01-25T05:01:10.135115+0000","last_became_active":"2023-01-25T05:01:09.071043+0000","last_became_peered":"2023-01-25T05:01:09.071043+0000","last_unstale":"2023-01-25T05:01:10.135115+0000","last_undegraded":"2023-01-25T05:01:10.135115+0000","last_fullsized":"2023-01-25T05:01:10.135115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:28.945742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.228535+0000","last_change":"2023-01-25T05:01:08.972545+0000","last_active":"2023-01-25T05:01:10.228535+0000","last_peered":"2023-01-25T05:01:10.228535+0000","last_clean":"2023-01-25T05:01:10.228535+0000","last_became_active":"2023-01-25T05:01:08.972336+0000","last_became_peered":"2023-01-25T05:01:08.972336+0000","last_unstale":"2023-01-25T05:01:10.228535+0000","last_undegraded":"2023-01-25T05:01:10.228535+0000","last_fullsized":"2023-01-25T05:01:10.228535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:18.753500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'104","reported_seq":147,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:12.675893+0000","last_change":"2023-01-25T05:01:05.234766+0000","last_active":"2023-01-25T05:01:12.675893+0000","last_peered":"2023-01-25T05:01:12.675893+0000","last_clean":"2023-01-25T05:01:12.675893+0000","last_became_active":"2023-01-25T05:01:05.133831+0000","last_became_peered":"2023-01-25T05:01:05.133831+0000","last_unstale":"2023-01-25T05:01:12.675893+0000","last_undegraded":"2023-01-25T05:01:12.675893+0000","last_fullsized":"2023-01-25T05:01:12.675893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:03.938898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:03.938898+0000","last_clean_scrub_stamp":"2023-01-25T05:01:03.938898+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:06.861570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":89,"num_write":143,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.188336+0000","last_change":"2023-01-25T05:01:09.070912+0000","last_active":"2023-01-25T05:01:10.188336+0000","last_peered":"2023-01-25T05:01:10.188336+0000","last_clean":"2023-01-25T05:01:10.188336+0000","last_became_active":"2023-01-25T05:01:09.070612+0000","last_became_peered":"2023-01-25T05:01:09.070612+0000","last_unstale":"2023-01-25T05:01:10.188336+0000","last_undegraded":"2023-01-25T05:01:10.188336+0000","last_fullsized":"2023-01-25T05:01:10.188336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:27.458590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.149001+0000","last_change":"2023-01-25T05:01:09.871074+0000","last_active":"2023-01-25T05:01:10.149001+0000","last_peered":"2023-01-25T05:01:10.149001+0000","last_clean":"2023-01-25T05:01:10.149001+0000","last_became_active":"2023-01-25T05:01:09.870775+0000","last_became_peered":"2023-01-25T05:01:09.870775+0000","last_unstale":"2023-01-25T05:01:10.149001+0000","last_undegraded":"2023-01-25T05:01:10.149001+0000","last_fullsized":"2023-01-25T05:01:10.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:27.378655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:10.188289+0000","last_change":"2023-01-25T05:01:09.071011+0000","last_active":"2023-01-25T05:01:10.188289+0000","last_peered":"2023-01-25T05:01:10.188289+0000","last_clean":"2023-01-25T05:01:10.188289+0000","last_became_active":"2023-01-25T05:01:09.070621+0000","last_became_peered":"2023-01-25T05:01:09.070621+0000","last_unstale":"2023-01-25T05:01:10.188289+0000","last_undegraded":"2023-01-25T05:01:10.188289+0000","last_fullsized":"2023-01-25T05:01:10.188289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:15.857186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-01-25T05:01:09.986418+0000","last_change":"2023-01-25T05:01:09.072711+0000","last_active":"2023-01-25T05:01:09.986418+0000","last_peered":"2023-01-25T05:01:09.986418+0000","last_clean":"2023-01-25T05:01:09.986418+0000","last_became_active":"2023-01-25T05:01:09.072473+0000","last_became_peered":"2023-01-25T05:01:09.072473+0000","last_unstale":"2023-01-25T05:01:09.986418+0000","last_undegraded":"2023-01-25T05:01:09.986418+0000","last_fullsized":"2023-01-25T05:01:09.986418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T05:01:07.955071+0000","last_clean_scrub_stamp":"2023-01-25T05:01:07.955071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T05:02:26.900431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":89,"num_write":143,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798208,"kb_used_data":92902320,"kb_used_omap":108,"kb_used_meta":0,"kb_avail":92901172,"statfs":{"total":95948165120,"available":95130800128,"internally_reserved":1175552,"allocated":95131975680,"data_stored":95131975680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":111320,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,0,0,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795856,"kb_used_data":92903560,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903524,"statfs":{"total":95948165120,"available":95133208576,"internally_reserved":36864,"allocated":95133245440,"data_stored":95133245440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43717,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":74.756}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795860,"kb_used_data":92903556,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92903520,"statfs":{"total":95948165120,"available":95133204480,"internally_reserved":36864,"allocated":95133241344,"data_stored":95133241344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":54387,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798196,"kb_used_data":92902324,"kb_used_omap":105,"kb_used_meta":0,"kb_avail":92901184,"statfs":{"total":95948165120,"available":95130812416,"internally_reserved":1167360,"allocated":95131979776,"data_stored":95131979776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":107576,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":1,"commit_latency_ns":0,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.664000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795844,"kb_used_data":92903556,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903536,"statfs":{"total":95948165120,"available":95133220864,"internally_reserved":20480,"allocated":95133241344,"data_stored":95133241344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43717,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.038}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.4290000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795896,"kb_used_data":92903516,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":92903484,"statfs":{"total":95948165120,"available":95133167616,"internally_reserved":32768,"allocated":95133200384,"data_stored":95133200384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38417,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.6289999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.282}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795704,"kb_used_data":92903684,"kb_used_omap":32,"kb_used_meta":0,"kb_avail":92903676,"statfs":{"total":95948165120,"available":95133364224,"internally_reserved":8192,"allocated":95133372416,"data_stored":95133372416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33085,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":31.378}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795848,"kb_used_data":92903556,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903532,"statfs":{"total":95948165120,"available":95133216768,"internally_reserved":24576,"allocated":95133241344,"data_stored":95133241344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43761,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]}],"pool_statfs":[]}} 2023-01-25T05:01:16.619 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T05:01:16.619 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-01-25T05:01:16.930 INFO:teuthology.orchestra.run.smithi135.stdout:epoch 21 2023-01-25T05:01:16.931 INFO:teuthology.orchestra.run.smithi135.stdout:fsid 8729840b-5c6f-4675-8457-dfb384ce25b3 2023-01-25T05:01:16.931 INFO:teuthology.orchestra.run.smithi135.stdout:created 2023-01-25T05:00:56.460333+0000 2023-01-25T05:01:16.931 INFO:teuthology.orchestra.run.smithi135.stdout:modified 2023-01-25T05:01:15.254254+0000 2023-01-25T05:01:16.931 INFO:teuthology.orchestra.run.smithi135.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-01-25T05:01:16.932 INFO:teuthology.orchestra.run.smithi135.stdout:crush_version 5 2023-01-25T05:01:16.932 INFO:teuthology.orchestra.run.smithi135.stdout:full_ratio 0.95 2023-01-25T05:01:16.932 INFO:teuthology.orchestra.run.smithi135.stdout:backfillfull_ratio 0.9 2023-01-25T05:01:16.932 INFO:teuthology.orchestra.run.smithi135.stdout:nearfull_ratio 0.85 2023-01-25T05:01:16.933 INFO:teuthology.orchestra.run.smithi135.stdout:require_min_compat_client luminous 2023-01-25T05:01:16.933 INFO:teuthology.orchestra.run.smithi135.stdout:min_compat_client jewel 2023-01-25T05:01:16.933 INFO:teuthology.orchestra.run.smithi135.stdout:require_osd_release reef 2023-01-25T05:01:16.933 INFO:teuthology.orchestra.run.smithi135.stdout:stretch_mode_enabled false 2023-01-25T05:01:16.933 INFO:teuthology.orchestra.run.smithi135.stdout:pool 1 '.mgr' replicated size 2 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2023-01-25T05:01:16.934 INFO:teuthology.orchestra.run.smithi135.stdout:pool 2 'rbd' replicated size 2 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 application rbd 2023-01-25T05:01:16.934 INFO:teuthology.orchestra.run.smithi135.stdout:max_osd 8 2023-01-25T05:01:16.934 INFO:teuthology.orchestra.run.smithi135.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6812/88726,v1:172.21.15.135:6814/88726] [v2:172.21.15.135:6817/88726,v1:172.21.15.135:6819/88726] exists,up 721fadcb-cd61-46a6-a18b-73833eeff96f 2023-01-25T05:01:16.934 INFO:teuthology.orchestra.run.smithi135.stdout:osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6808/88727,v1:172.21.15.135:6809/88727] [v2:172.21.15.135:6810/88727,v1:172.21.15.135:6811/88727] exists,up b6793ed6-09d7-436d-a063-aa3a97463a92 2023-01-25T05:01:16.935 INFO:teuthology.orchestra.run.smithi135.stdout:osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6816/88728,v1:172.21.15.135:6825/88728] [v2:172.21.15.135:6826/88728,v1:172.21.15.135:6827/88728] exists,up 5b30b63e-a29f-4ab5-813e-453ced6184dc 2023-01-25T05:01:16.935 INFO:teuthology.orchestra.run.smithi135.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6800/88725,v1:172.21.15.135:6801/88725] [v2:172.21.15.135:6802/88725,v1:172.21.15.135:6803/88725] exists,up e64ff366-6511-49aa-9562-32da645862e6 2023-01-25T05:01:16.935 INFO:teuthology.orchestra.run.smithi135.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6814/88693,v1:172.21.15.178:6816/88693] [v2:172.21.15.178:6818/88693,v1:172.21.15.178:6820/88693] exists,up 2a56593a-39d8-4bac-b749-902fd6a2f842 2023-01-25T05:01:16.935 INFO:teuthology.orchestra.run.smithi135.stdout:osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6808/88692,v1:172.21.15.178:6809/88692] [v2:172.21.15.178:6810/88692,v1:172.21.15.178:6811/88692] exists,up 40596a50-ff52-43ab-b737-cba69586855e 2023-01-25T05:01:16.935 INFO:teuthology.orchestra.run.smithi135.stdout:osd.6 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6819/88694,v1:172.21.15.178:6821/88694] [v2:172.21.15.178:6823/88694,v1:172.21.15.178:6825/88694] exists,up 44b2e2c2-d15c-4e13-90e5-7038a4ae907e 2023-01-25T05:01:16.936 INFO:teuthology.orchestra.run.smithi135.stdout:osd.7 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6800/88691,v1:172.21.15.178:6801/88691] [v2:172.21.15.178:6802/88691,v1:172.21.15.178:6803/88691] exists,up d864c502-0546-4daa-b8b0-79228e0e61a5 2023-01-25T05:01:16.944 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6812/88726,v1:172.21.15.135:6814/88726] [v2:172.21.15.135:6817/88726,v1:172.21.15.135:6819/88726] exists,up 721fadcb-cd61-46a6-a18b-73833eeff96f', 'osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6808/88727,v1:172.21.15.135:6809/88727] [v2:172.21.15.135:6810/88727,v1:172.21.15.135:6811/88727] exists,up b6793ed6-09d7-436d-a063-aa3a97463a92', 'osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6816/88728,v1:172.21.15.135:6825/88728] [v2:172.21.15.135:6826/88728,v1:172.21.15.135:6827/88728] exists,up 5b30b63e-a29f-4ab5-813e-453ced6184dc', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6800/88725,v1:172.21.15.135:6801/88725] [v2:172.21.15.135:6802/88725,v1:172.21.15.135:6803/88725] exists,up e64ff366-6511-49aa-9562-32da645862e6', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6814/88693,v1:172.21.15.178:6816/88693] [v2:172.21.15.178:6818/88693,v1:172.21.15.178:6820/88693] exists,up 2a56593a-39d8-4bac-b749-902fd6a2f842', 'osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6808/88692,v1:172.21.15.178:6809/88692] [v2:172.21.15.178:6810/88692,v1:172.21.15.178:6811/88692] exists,up 40596a50-ff52-43ab-b737-cba69586855e', 'osd.6 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6819/88694,v1:172.21.15.178:6821/88694] [v2:172.21.15.178:6823/88694,v1:172.21.15.178:6825/88694] exists,up 44b2e2c2-d15c-4e13-90e5-7038a4ae907e', 'osd.7 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6800/88691,v1:172.21.15.178:6801/88691] [v2:172.21.15.178:6802/88691,v1:172.21.15.178:6803/88691] exists,up d864c502-0546-4daa-b8b0-79228e0e61a5'] 2023-01-25T05:01:16.945 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-01-25T05:01:17.065 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2023-01-25T05:01:17.066 INFO:teuthology.orchestra.run.smithi135.stdout: "name": "mon.a", 2023-01-25T05:01:17.066 INFO:teuthology.orchestra.run.smithi135.stdout: "cluster": "ceph", 2023-01-25T05:01:17.066 INFO:teuthology.orchestra.run.smithi135.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-01-25T05:01:17.067 INFO:teuthology.orchestra.run.smithi135.stdout: "admin_socket_mode": "", 2023-01-25T05:01:17.067 INFO:teuthology.orchestra.run.smithi135.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-01-25T05:01:17.067 INFO:teuthology.orchestra.run.smithi135.stdout: "auth_client_required": "cephx, none", 2023-01-25T05:01:17.068 INFO:teuthology.orchestra.run.smithi135.stdout: "auth_cluster_required": "cephx", 2023-01-25T05:01:17.068 INFO:teuthology.orchestra.run.smithi135.stdout: "auth_debug": "true", 2023-01-25T05:01:17.068 INFO:teuthology.orchestra.run.smithi135.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-01-25T05:01:17.068 INFO:teuthology.orchestra.run.smithi135.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-01-25T05:01:17.068 INFO:teuthology.orchestra.run.smithi135.stdout: "auth_service_required": "cephx", 2023-01-25T05:01:17.069 INFO:teuthology.orchestra.run.smithi135.stdout: "auth_service_ticket_ttl": "240.000000", 2023-01-25T05:01:17.069 INFO:teuthology.orchestra.run.smithi135.stdout: "auth_supported": "cephx", 2023-01-25T05:01:17.069 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_aio": "true", 2023-01-25T05:01:17.069 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_aio_max_queue_depth": "1024", 2023-01-25T05:01:17.070 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_aio_poll_ms": "250", 2023-01-25T05:01:17.070 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_aio_reap_max": "16", 2023-01-25T05:01:17.070 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_async_discard": "false", 2023-01-25T05:01:17.070 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_block_size": "4096", 2023-01-25T05:01:17.070 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_debug_aio": "false", 2023-01-25T05:01:17.071 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-01-25T05:01:17.071 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-01-25T05:01:17.071 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_debug_inflight_ios": "false", 2023-01-25T05:01:17.071 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_enable_discard": "false", 2023-01-25T05:01:17.072 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_flock_retry": "3", 2023-01-25T05:01:17.072 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_flock_retry_interval": "0.100000", 2023-01-25T05:01:17.072 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_inject_crash": "0", 2023-01-25T05:01:17.072 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_inject_crash_flush_delay": "2", 2023-01-25T05:01:17.072 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_ioring": "false", 2023-01-25T05:01:17.073 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_ioring_hipri": "false", 2023-01-25T05:01:17.073 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_ioring_sqthread_poll": "false", 2023-01-25T05:01:17.073 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-01-25T05:01:17.073 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_read_buffer_alignment": "4096", 2023-01-25T05:01:17.074 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-01-25T05:01:17.074 INFO:teuthology.orchestra.run.smithi135.stdout: "bdev_type": "", 2023-01-25T05:01:17.074 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_alloc_size": "1048576", 2023-01-25T05:01:17.074 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_allocator": "hybrid", 2023-01-25T05:01:17.075 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_buffered_io": "true", 2023-01-25T05:01:17.075 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_check_for_zeros": "false", 2023-01-25T05:01:17.075 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_check_volume_selector_often": "false", 2023-01-25T05:01:17.075 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-01-25T05:01:17.075 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_compact_log_sync": "false", 2023-01-25T05:01:17.076 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-01-25T05:01:17.076 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_log_compact_min_size": "16777216", 2023-01-25T05:01:17.076 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_log_replay_check_allocations": "true", 2023-01-25T05:01:17.076 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_max_log_runway": "4194304", 2023-01-25T05:01:17.076 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_max_prefetch": "1048576", 2023-01-25T05:01:17.077 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_min_flush_size": "524288", 2023-01-25T05:01:17.077 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_min_log_runway": "1048576", 2023-01-25T05:01:17.077 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_replay_recovery": "false", 2023-01-25T05:01:17.078 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-01-25T05:01:17.078 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_shared_alloc_size": "65536", 2023-01-25T05:01:17.078 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_sync_write": "false", 2023-01-25T05:01:17.078 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-01-25T05:01:17.079 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-01-25T05:01:17.079 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-01-25T05:01:17.079 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_allocation_from_file": "true", 2023-01-25T05:01:17.079 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_allocator": "hybrid", 2023-01-25T05:01:17.079 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-01-25T05:01:17.080 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-01-25T05:01:17.080 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-01-25T05:01:17.080 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-01-25T05:01:17.080 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-01-25T05:01:17.080 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-01-25T05:01:17.081 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blobid_prealloc": "10240", 2023-01-25T05:01:17.081 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_block_create": "true", 2023-01-25T05:01:17.081 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_block_db_create": "false", 2023-01-25T05:01:17.081 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_block_db_path": "", 2023-01-25T05:01:17.081 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_block_db_size": "0", 2023-01-25T05:01:17.082 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_block_path": "", 2023-01-25T05:01:17.082 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_block_preallocate_file": "false", 2023-01-25T05:01:17.082 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_block_size": "107374182400", 2023-01-25T05:01:17.082 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_block_wal_create": "false", 2023-01-25T05:01:17.082 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_block_wal_path": "", 2023-01-25T05:01:17.083 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_block_wal_size": "100663296", 2023-01-25T05:01:17.083 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_bluefs": "true", 2023-01-25T05:01:17.083 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-01-25T05:01:17.083 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_bluefs_env_mirror": "false", 2023-01-25T05:01:17.083 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-01-25T05:01:17.084 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-01-25T05:01:17.084 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T05:01:17.084 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T05:01:17.084 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-01-25T05:01:17.084 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T05:01:17.085 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_autotune": "true", 2023-01-25T05:01:17.085 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-01-25T05:01:17.085 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-01-25T05:01:17.085 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-01-25T05:01:17.085 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-01-25T05:01:17.086 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_size": "0", 2023-01-25T05:01:17.086 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-01-25T05:01:17.086 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-01-25T05:01:17.086 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-01-25T05:01:17.086 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-01-25T05:01:17.087 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_type": "2q", 2023-01-25T05:01:17.087 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-01-25T05:01:17.087 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_clone_cow": "true", 2023-01-25T05:01:17.087 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_compression_algorithm": "snappy", 2023-01-25T05:01:17.088 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_compression_max_blob_size": "0", 2023-01-25T05:01:17.088 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-01-25T05:01:17.088 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-01-25T05:01:17.088 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_compression_min_blob_size": "0", 2023-01-25T05:01:17.088 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-01-25T05:01:17.089 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-01-25T05:01:17.089 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_compression_mode": "none", 2023-01-25T05:01:17.089 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-01-25T05:01:17.089 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_csum_type": "crc32c", 2023-01-25T05:01:17.089 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_enforce_settings": "default", 2023-01-25T05:01:17.090 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_freelist": "false", 2023-01-25T05:01:17.090 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_fsck_abort": "false", 2023-01-25T05:01:17.090 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2023-01-25T05:01:17.090 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_inject_bug21040": "false", 2023-01-25T05:01:17.090 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-01-25T05:01:17.091 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_inject_read_err": "false", 2023-01-25T05:01:17.091 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_legacy_omap": "false", 2023-01-25T05:01:17.091 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-01-25T05:01:17.091 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-01-25T05:01:17.091 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-01-25T05:01:17.092 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-01-25T05:01:17.092 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_prefill": "0.000000", 2023-01-25T05:01:17.092 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-01-25T05:01:17.092 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-01-25T05:01:17.093 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-01-25T05:01:17.093 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_small_allocations": "0", 2023-01-25T05:01:17.093 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-01-25T05:01:17.093 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_default_buffered_read": "true", 2023-01-25T05:01:17.093 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_default_buffered_write": "false", 2023-01-25T05:01:17.094 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_deferred_batch_ops": "0", 2023-01-25T05:01:17.094 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-01-25T05:01:17.094 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-01-25T05:01:17.094 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-01-25T05:01:17.094 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-01-25T05:01:17.095 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-01-25T05:01:17.095 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-01-25T05:01:17.095 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-01-25T05:01:17.095 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fail_eio": "false", 2023-01-25T05:01:17.095 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-01-25T05:01:17.096 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-01-25T05:01:17.096 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-01-25T05:01:17.096 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-01-25T05:01:17.096 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_on_mkfs": "true", 2023-01-25T05:01:17.097 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-01-25T05:01:17.097 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_on_mount": "false", 2023-01-25T05:01:17.097 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-01-25T05:01:17.097 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_on_umount": "false", 2023-01-25T05:01:17.097 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-01-25T05:01:17.098 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-01-25T05:01:17.098 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-01-25T05:01:17.098 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-01-25T05:01:17.098 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-01-25T05:01:17.098 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-01-25T05:01:17.099 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-01-25T05:01:17.099 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-01-25T05:01:17.099 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_ignore_data_csum": "false", 2023-01-25T05:01:17.099 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-01-25T05:01:17.099 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_kvbackend": "rocksdb", 2023-01-25T05:01:17.099 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-01-25T05:01:17.100 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-01-25T05:01:17.100 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_log_op_age": "5.000000", 2023-01-25T05:01:17.100 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_max_alloc_size": "0", 2023-01-25T05:01:17.100 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_max_blob_size": "0", 2023-01-25T05:01:17.100 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-01-25T05:01:17.101 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-01-25T05:01:17.101 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_max_defer_interval": "3.000000", 2023-01-25T05:01:17.101 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_max_deferred_txc": "32", 2023-01-25T05:01:17.101 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_min_alloc_size": "0", 2023-01-25T05:01:17.101 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-01-25T05:01:17.102 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-01-25T05:01:17.102 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_nid_prealloc": "1024", 2023-01-25T05:01:17.102 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_prefer_deferred_size": "0", 2023-01-25T05:01:17.102 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-01-25T05:01:17.102 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-01-25T05:01:17.103 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_qfsck_on_mount": "true", 2023-01-25T05:01:17.103 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_retry_disk_reads": "3", 2023-01-25T05:01:17.103 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_rocksdb_cf": "true", 2023-01-25T05:01:17.103 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2023-01-25T05:01:17.103 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=128,min_write_buffer_number_to_merge=16,compaction_style=kCompactionStyleLevel,write_buffer_size=8388608,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0,ttl=21600", 2023-01-25T05:01:17.104 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_rocksdb_options_annex": "", 2023-01-25T05:01:17.104 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_spdk_coremask": "0x1", 2023-01-25T05:01:17.104 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_spdk_io_sleep": "5", 2023-01-25T05:01:17.104 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_spdk_max_io_completion": "0", 2023-01-25T05:01:17.104 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_spdk_mem": "512", 2023-01-25T05:01:17.105 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_sync_submit_transaction": "false", 2023-01-25T05:01:17.105 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_throttle_bytes": "67108864", 2023-01-25T05:01:17.105 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_throttle_cost_per_io": "0", 2023-01-25T05:01:17.105 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-01-25T05:01:17.105 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-01-25T05:01:17.106 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-01-25T05:01:17.106 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-01-25T05:01:17.106 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_tracing": "false", 2023-01-25T05:01:17.106 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-01-25T05:01:17.106 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-01-25T05:01:17.107 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_volume_selection_reserved": "0", 2023-01-25T05:01:17.107 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-01-25T05:01:17.107 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-01-25T05:01:17.107 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-01-25T05:01:17.107 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-01-25T05:01:17.107 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-01-25T05:01:17.108 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-01-25T05:01:17.108 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_zero_block_detection": "false", 2023-01-25T05:01:17.108 INFO:teuthology.orchestra.run.smithi135.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-01-25T05:01:17.108 INFO:teuthology.orchestra.run.smithi135.stdout: "cephfs_mirror_action_update_interval": "2", 2023-01-25T05:01:17.108 INFO:teuthology.orchestra.run.smithi135.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-01-25T05:01:17.109 INFO:teuthology.orchestra.run.smithi135.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-01-25T05:01:17.109 INFO:teuthology.orchestra.run.smithi135.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-01-25T05:01:17.109 INFO:teuthology.orchestra.run.smithi135.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-01-25T05:01:17.109 INFO:teuthology.orchestra.run.smithi135.stdout: "cephfs_mirror_mount_timeout": "10", 2023-01-25T05:01:17.109 INFO:teuthology.orchestra.run.smithi135.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-01-25T05:01:17.109 INFO:teuthology.orchestra.run.smithi135.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-01-25T05:01:17.110 INFO:teuthology.orchestra.run.smithi135.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-01-25T05:01:17.110 INFO:teuthology.orchestra.run.smithi135.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-01-25T05:01:17.110 INFO:teuthology.orchestra.run.smithi135.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-01-25T05:01:17.110 INFO:teuthology.orchestra.run.smithi135.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-01-25T05:01:17.110 INFO:teuthology.orchestra.run.smithi135.stdout: "cephx_cluster_require_signatures": "false", 2023-01-25T05:01:17.111 INFO:teuthology.orchestra.run.smithi135.stdout: "cephx_cluster_require_version": "2", 2023-01-25T05:01:17.111 INFO:teuthology.orchestra.run.smithi135.stdout: "cephx_require_signatures": "false", 2023-01-25T05:01:17.111 INFO:teuthology.orchestra.run.smithi135.stdout: "cephx_require_version": "2", 2023-01-25T05:01:17.111 INFO:teuthology.orchestra.run.smithi135.stdout: "cephx_service_require_signatures": "false", 2023-01-25T05:01:17.111 INFO:teuthology.orchestra.run.smithi135.stdout: "cephx_service_require_version": "2", 2023-01-25T05:01:17.111 INFO:teuthology.orchestra.run.smithi135.stdout: "cephx_sign_messages": "true", 2023-01-25T05:01:17.112 INFO:teuthology.orchestra.run.smithi135.stdout: "chdir": "", 2023-01-25T05:01:17.112 INFO:teuthology.orchestra.run.smithi135.stdout: "client_acl_type": "", 2023-01-25T05:01:17.112 INFO:teuthology.orchestra.run.smithi135.stdout: "client_asio_thread_count": "2", 2023-01-25T05:01:17.112 INFO:teuthology.orchestra.run.smithi135.stdout: "client_cache_mid": "0.750000", 2023-01-25T05:01:17.112 INFO:teuthology.orchestra.run.smithi135.stdout: "client_cache_size": "16384", 2023-01-25T05:01:17.113 INFO:teuthology.orchestra.run.smithi135.stdout: "client_caps_release_delay": "5", 2023-01-25T05:01:17.113 INFO:teuthology.orchestra.run.smithi135.stdout: "client_check_pool_perm": "true", 2023-01-25T05:01:17.113 INFO:teuthology.orchestra.run.smithi135.stdout: "client_collect_and_send_global_metrics": "false", 2023-01-25T05:01:17.113 INFO:teuthology.orchestra.run.smithi135.stdout: "client_debug_force_sync_read": "false", 2023-01-25T05:01:17.113 INFO:teuthology.orchestra.run.smithi135.stdout: "client_debug_getattr_caps": "false", 2023-01-25T05:01:17.114 INFO:teuthology.orchestra.run.smithi135.stdout: "client_debug_inject_tick_delay": "0", 2023-01-25T05:01:17.114 INFO:teuthology.orchestra.run.smithi135.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-01-25T05:01:17.114 INFO:teuthology.orchestra.run.smithi135.stdout: "client_die_on_failed_remount": "false", 2023-01-25T05:01:17.114 INFO:teuthology.orchestra.run.smithi135.stdout: "client_dirsize_rbytes": "true", 2023-01-25T05:01:17.114 INFO:teuthology.orchestra.run.smithi135.stdout: "client_force_lazyio": "false", 2023-01-25T05:01:17.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client_fs": "", 2023-01-25T05:01:17.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client_inject_fixed_oldest_tid": "false", 2023-01-25T05:01:17.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client_inject_release_failure": "false", 2023-01-25T05:01:17.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client_max_inline_size": "4096", 2023-01-25T05:01:17.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client_mds_namespace": "", 2023-01-25T05:01:17.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client_metadata": "", 2023-01-25T05:01:17.116 INFO:teuthology.orchestra.run.smithi135.stdout: "client_mount_gid": "-1", 2023-01-25T05:01:17.116 INFO:teuthology.orchestra.run.smithi135.stdout: "client_mount_timeout": "300", 2023-01-25T05:01:17.116 INFO:teuthology.orchestra.run.smithi135.stdout: "client_mount_uid": "-1", 2023-01-25T05:01:17.116 INFO:teuthology.orchestra.run.smithi135.stdout: "client_mountpoint": "/", 2023-01-25T05:01:17.117 INFO:teuthology.orchestra.run.smithi135.stdout: "client_notify_timeout": "10", 2023-01-25T05:01:17.117 INFO:teuthology.orchestra.run.smithi135.stdout: "client_oc": "true", 2023-01-25T05:01:17.117 INFO:teuthology.orchestra.run.smithi135.stdout: "client_oc_max_dirty": "104857600", 2023-01-25T05:01:17.117 INFO:teuthology.orchestra.run.smithi135.stdout: "client_oc_max_dirty_age": "5.000000", 2023-01-25T05:01:17.117 INFO:teuthology.orchestra.run.smithi135.stdout: "client_oc_max_objects": "1000", 2023-01-25T05:01:17.118 INFO:teuthology.orchestra.run.smithi135.stdout: "client_oc_size": "209715200", 2023-01-25T05:01:17.118 INFO:teuthology.orchestra.run.smithi135.stdout: "client_oc_target_dirty": "8388608", 2023-01-25T05:01:17.118 INFO:teuthology.orchestra.run.smithi135.stdout: "client_permissions": "true", 2023-01-25T05:01:17.118 INFO:teuthology.orchestra.run.smithi135.stdout: "client_quota": "true", 2023-01-25T05:01:17.118 INFO:teuthology.orchestra.run.smithi135.stdout: "client_quota_df": "true", 2023-01-25T05:01:17.119 INFO:teuthology.orchestra.run.smithi135.stdout: "client_readahead_max_bytes": "0", 2023-01-25T05:01:17.119 INFO:teuthology.orchestra.run.smithi135.stdout: "client_readahead_max_periods": "4", 2023-01-25T05:01:17.119 INFO:teuthology.orchestra.run.smithi135.stdout: "client_readahead_min": "131072", 2023-01-25T05:01:17.119 INFO:teuthology.orchestra.run.smithi135.stdout: "client_reconnect_stale": "false", 2023-01-25T05:01:17.119 INFO:teuthology.orchestra.run.smithi135.stdout: "client_shutdown_timeout": "30", 2023-01-25T05:01:17.120 INFO:teuthology.orchestra.run.smithi135.stdout: "client_snapdir": ".snap", 2023-01-25T05:01:17.120 INFO:teuthology.orchestra.run.smithi135.stdout: "client_tick_interval": "1", 2023-01-25T05:01:17.120 INFO:teuthology.orchestra.run.smithi135.stdout: "client_trace": "", 2023-01-25T05:01:17.120 INFO:teuthology.orchestra.run.smithi135.stdout: "client_try_dentry_invalidate": "false", 2023-01-25T05:01:17.120 INFO:teuthology.orchestra.run.smithi135.stdout: "client_use_faked_inos": "false", 2023-01-25T05:01:17.121 INFO:teuthology.orchestra.run.smithi135.stdout: "client_use_random_mds": "false", 2023-01-25T05:01:17.121 INFO:teuthology.orchestra.run.smithi135.stdout: "clog_to_graylog": "false", 2023-01-25T05:01:17.121 INFO:teuthology.orchestra.run.smithi135.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-01-25T05:01:17.121 INFO:teuthology.orchestra.run.smithi135.stdout: "clog_to_graylog_port": "12201", 2023-01-25T05:01:17.121 INFO:teuthology.orchestra.run.smithi135.stdout: "clog_to_monitors": "default=true", 2023-01-25T05:01:17.121 INFO:teuthology.orchestra.run.smithi135.stdout: "clog_to_syslog": "false", 2023-01-25T05:01:17.122 INFO:teuthology.orchestra.run.smithi135.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-01-25T05:01:17.122 INFO:teuthology.orchestra.run.smithi135.stdout: "clog_to_syslog_level": "info", 2023-01-25T05:01:17.122 INFO:teuthology.orchestra.run.smithi135.stdout: "cluster_addr": "-", 2023-01-25T05:01:17.122 INFO:teuthology.orchestra.run.smithi135.stdout: "cluster_network": "", 2023-01-25T05:01:17.122 INFO:teuthology.orchestra.run.smithi135.stdout: "cluster_network_interface": "", 2023-01-25T05:01:17.123 INFO:teuthology.orchestra.run.smithi135.stdout: "compressor_zlib_isal": "false", 2023-01-25T05:01:17.123 INFO:teuthology.orchestra.run.smithi135.stdout: "compressor_zlib_level": "5", 2023-01-25T05:01:17.123 INFO:teuthology.orchestra.run.smithi135.stdout: "compressor_zlib_winsize": "-15", 2023-01-25T05:01:17.123 INFO:teuthology.orchestra.run.smithi135.stdout: "compressor_zstd_level": "1", 2023-01-25T05:01:17.123 INFO:teuthology.orchestra.run.smithi135.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-01-25T05:01:17.124 INFO:teuthology.orchestra.run.smithi135.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-01-25T05:01:17.124 INFO:teuthology.orchestra.run.smithi135.stdout: "crimson_alien_op_num_threads": "6", 2023-01-25T05:01:17.124 INFO:teuthology.orchestra.run.smithi135.stdout: "crimson_alien_thread_cpu_cores": "", 2023-01-25T05:01:17.124 INFO:teuthology.orchestra.run.smithi135.stdout: "crimson_osd_obc_lru_size": "10", 2023-01-25T05:01:17.124 INFO:teuthology.orchestra.run.smithi135.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-01-25T05:01:17.125 INFO:teuthology.orchestra.run.smithi135.stdout: "crimson_seastar_smp": "1", 2023-01-25T05:01:17.125 INFO:teuthology.orchestra.run.smithi135.stdout: "crush_location": "", 2023-01-25T05:01:17.125 INFO:teuthology.orchestra.run.smithi135.stdout: "crush_location_hook": "", 2023-01-25T05:01:17.125 INFO:teuthology.orchestra.run.smithi135.stdout: "crush_location_hook_timeout": "10", 2023-01-25T05:01:17.125 INFO:teuthology.orchestra.run.smithi135.stdout: "daemonize": "false", 2023-01-25T05:01:17.126 INFO:teuthology.orchestra.run.smithi135.stdout: "daos_pool": "tank", 2023-01-25T05:01:17.126 INFO:teuthology.orchestra.run.smithi135.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2023-01-25T05:01:17.126 INFO:teuthology.orchestra.run.smithi135.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2023-01-25T05:01:17.126 INFO:teuthology.orchestra.run.smithi135.stdout: "dbstore_db_name_prefix": "dbstore", 2023-01-25T05:01:17.126 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_alienstore": "0/5", 2023-01-25T05:01:17.126 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_allow_any_pool_priority": "false", 2023-01-25T05:01:17.127 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_asok": "1/5", 2023-01-25T05:01:17.127 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_asok_assert_abort": "false", 2023-01-25T05:01:17.127 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_asserts_on_shutdown": "true", 2023-01-25T05:01:17.127 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_auth": "20/20", 2023-01-25T05:01:17.127 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_bdev": "1/3", 2023-01-25T05:01:17.128 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_bluefs": "1/5", 2023-01-25T05:01:17.128 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_bluestore": "1/5", 2023-01-25T05:01:17.128 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_buffer": "0/1", 2023-01-25T05:01:17.128 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_ceph_exporter": "1/5", 2023-01-25T05:01:17.128 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_cephfs_mirror": "0/5", 2023-01-25T05:01:17.129 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_cephsqlite": "0/5", 2023-01-25T05:01:17.129 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_client": "0/5", 2023-01-25T05:01:17.129 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_compressor": "1/5", 2023-01-25T05:01:17.129 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_context": "0/1", 2023-01-25T05:01:17.129 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_crush": "1/1", 2023-01-25T05:01:17.130 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_crypto": "1/5", 2023-01-25T05:01:17.130 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_cyanstore": "0/5", 2023-01-25T05:01:17.130 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_deliberately_leak_memory": "false", 2023-01-25T05:01:17.130 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_disable_randomized_ping": "false", 2023-01-25T05:01:17.130 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_dpdk": "1/5", 2023-01-25T05:01:17.131 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_eventtrace": "1/5", 2023-01-25T05:01:17.131 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_filer": "0/1", 2023-01-25T05:01:17.131 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_filestore": "1/3", 2023-01-25T05:01:17.131 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_finisher": "1/1", 2023-01-25T05:01:17.131 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_fuse": "1/5", 2023-01-25T05:01:17.132 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_heartbeat_testing_span": "0", 2023-01-25T05:01:17.132 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_heartbeatmap": "1/5", 2023-01-25T05:01:17.132 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_immutable_obj_cache": "0/5", 2023-01-25T05:01:17.132 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_javaclient": "1/5", 2023-01-25T05:01:17.132 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_journal": "1/3", 2023-01-25T05:01:17.133 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_journaler": "0/5", 2023-01-25T05:01:17.133 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_kstore": "1/5", 2023-01-25T05:01:17.133 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_leveldb": "4/5", 2023-01-25T05:01:17.133 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_lockdep": "0/1", 2023-01-25T05:01:17.133 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_mclock": "1/5", 2023-01-25T05:01:17.134 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_mds": "1/5", 2023-01-25T05:01:17.134 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_mds_balancer": "1/5", 2023-01-25T05:01:17.134 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_mds_locker": "1/5", 2023-01-25T05:01:17.134 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_mds_log": "1/5", 2023-01-25T05:01:17.134 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_mds_log_expire": "1/5", 2023-01-25T05:01:17.135 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_mds_migrator": "1/5", 2023-01-25T05:01:17.135 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_memstore": "1/5", 2023-01-25T05:01:17.135 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_mgr": "2/5", 2023-01-25T05:01:17.135 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_mgrc": "1/5", 2023-01-25T05:01:17.136 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_mon": "20/20", 2023-01-25T05:01:17.136 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_monc": "0/10", 2023-01-25T05:01:17.136 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_ms": "1/1", 2023-01-25T05:01:17.136 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_none": "0/5", 2023-01-25T05:01:17.136 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_objclass": "0/5", 2023-01-25T05:01:17.137 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_objectcacher": "0/5", 2023-01-25T05:01:17.137 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_objecter": "0/1", 2023-01-25T05:01:17.137 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_optracker": "0/5", 2023-01-25T05:01:17.137 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_osd": "1/5", 2023-01-25T05:01:17.137 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_paxos": "20/20", 2023-01-25T05:01:17.138 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_perfcounter": "1/5", 2023-01-25T05:01:17.138 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_prioritycache": "1/5", 2023-01-25T05:01:17.138 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rados": "0/5", 2023-01-25T05:01:17.138 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rbd": "0/5", 2023-01-25T05:01:17.138 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rbd_mirror": "0/5", 2023-01-25T05:01:17.139 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rbd_pwl": "0/5", 2023-01-25T05:01:17.139 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rbd_replay": "0/5", 2023-01-25T05:01:17.139 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_refs": "0/0", 2023-01-25T05:01:17.139 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_reserver": "1/1", 2023-01-25T05:01:17.139 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rgw": "1/5", 2023-01-25T05:01:17.140 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rgw_access": "1/5", 2023-01-25T05:01:17.140 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rgw_datacache": "1/5", 2023-01-25T05:01:17.140 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rgw_dbstore": "1/5", 2023-01-25T05:01:17.140 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rgw_flight": "1/5", 2023-01-25T05:01:17.140 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rgw_sync": "1/5", 2023-01-25T05:01:17.141 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_rocksdb": "4/5", 2023-01-25T05:01:17.141 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore": "0/5", 2023-01-25T05:01:17.141 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_backref": "0/5", 2023-01-25T05:01:17.141 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_cache": "0/5", 2023-01-25T05:01:17.141 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_cleaner": "0/5", 2023-01-25T05:01:17.142 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_device": "0/5", 2023-01-25T05:01:17.142 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_epm": "0/5", 2023-01-25T05:01:17.142 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_fixedkv_tree": "0/5", 2023-01-25T05:01:17.142 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_journal": "0/5", 2023-01-25T05:01:17.142 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_lba": "0/5", 2023-01-25T05:01:17.143 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_odata": "0/5", 2023-01-25T05:01:17.143 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_omap": "0/5", 2023-01-25T05:01:17.143 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_onode": "0/5", 2023-01-25T05:01:17.143 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_t": "0/5", 2023-01-25T05:01:17.143 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_seastore_tm": "0/5", 2023-01-25T05:01:17.144 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_striper": "0/1", 2023-01-25T05:01:17.144 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_test": "0/5", 2023-01-25T05:01:17.144 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_throttle": "1/1", 2023-01-25T05:01:17.144 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_timer": "0/1", 2023-01-25T05:01:17.144 INFO:teuthology.orchestra.run.smithi135.stdout: "debug_tp": "0/5", 2023-01-25T05:01:17.145 INFO:teuthology.orchestra.run.smithi135.stdout: "device_failure_prediction_mode": "none", 2023-01-25T05:01:17.145 INFO:teuthology.orchestra.run.smithi135.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-01-25T05:01:17.145 INFO:teuthology.orchestra.run.smithi135.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-01-25T05:01:17.145 INFO:teuthology.orchestra.run.smithi135.stdout: "err_to_graylog": "false", 2023-01-25T05:01:17.145 INFO:teuthology.orchestra.run.smithi135.stdout: "err_to_journald": "false", 2023-01-25T05:01:17.146 INFO:teuthology.orchestra.run.smithi135.stdout: "err_to_stderr": "true", 2023-01-25T05:01:17.146 INFO:teuthology.orchestra.run.smithi135.stdout: "err_to_syslog": "false", 2023-01-25T05:01:17.146 INFO:teuthology.orchestra.run.smithi135.stdout: "event_tracing": "false", 2023-01-25T05:01:17.146 INFO:teuthology.orchestra.run.smithi135.stdout: "exporter_addr": "0.0.0.0", 2023-01-25T05:01:17.146 INFO:teuthology.orchestra.run.smithi135.stdout: "exporter_http_port": "9926", 2023-01-25T05:01:17.147 INFO:teuthology.orchestra.run.smithi135.stdout: "exporter_prio_limit": "5", 2023-01-25T05:01:17.147 INFO:teuthology.orchestra.run.smithi135.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-01-25T05:01:17.147 INFO:teuthology.orchestra.run.smithi135.stdout: "exporter_sort_metrics": "true", 2023-01-25T05:01:17.147 INFO:teuthology.orchestra.run.smithi135.stdout: "exporter_stats_period": "5", 2023-01-25T05:01:17.147 INFO:teuthology.orchestra.run.smithi135.stdout: "fake_statfs_for_testing": "0", 2023-01-25T05:01:17.148 INFO:teuthology.orchestra.run.smithi135.stdout: "fatal_signal_handlers": "true", 2023-01-25T05:01:17.148 INFO:teuthology.orchestra.run.smithi135.stdout: "filer_max_purge_ops": "10", 2023-01-25T05:01:17.148 INFO:teuthology.orchestra.run.smithi135.stdout: "filer_max_truncate_ops": "128", 2023-01-25T05:01:17.148 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_apply_finisher_threads": "1", 2023-01-25T05:01:17.148 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_blackhole": "false", 2023-01-25T05:01:17.149 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_btrfs_clone_range": "true", 2023-01-25T05:01:17.149 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_btrfs_snap": "true", 2023-01-25T05:01:17.149 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_caller_concurrency": "10", 2023-01-25T05:01:17.149 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_collect_device_partition_information": "true", 2023-01-25T05:01:17.149 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_commit_timeout": "600.000000", 2023-01-25T05:01:17.150 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_debug_inject_read_err": "false", 2023-01-25T05:01:17.150 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_debug_omap_check": "false", 2023-01-25T05:01:17.150 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_debug_random_read_err": "0.000000", 2023-01-25T05:01:17.150 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_debug_verify_split": "false", 2023-01-25T05:01:17.150 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_dump_file": "", 2023-01-25T05:01:17.151 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-01-25T05:01:17.151 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-01-25T05:01:17.151 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_fadvise": "true", 2023-01-25T05:01:17.151 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_fail_eio": "true", 2023-01-25T05:01:17.151 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_fd_cache_shards": "16", 2023-01-25T05:01:17.152 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_fd_cache_size": "128", 2023-01-25T05:01:17.152 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_fiemap": "false", 2023-01-25T05:01:17.152 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_fiemap_threshold": "4096", 2023-01-25T05:01:17.152 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-01-25T05:01:17.152 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_index_retry_probability": "0.000000", 2023-01-25T05:01:17.153 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_inject_stall": "0", 2023-01-25T05:01:17.153 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_journal_parallel": "false", 2023-01-25T05:01:17.153 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_journal_trailing": "false", 2023-01-25T05:01:17.153 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_journal_writeahead": "false", 2023-01-25T05:01:17.153 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_kill_at": "0", 2023-01-25T05:01:17.153 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-01-25T05:01:17.154 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_inline_xattr_size": "0", 2023-01-25T05:01:17.154 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-01-25T05:01:17.154 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-01-25T05:01:17.154 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-01-25T05:01:17.154 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_inline_xattrs": "0", 2023-01-25T05:01:17.155 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-01-25T05:01:17.155 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_inline_xattrs_other": "2", 2023-01-25T05:01:17.155 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-01-25T05:01:17.155 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_sync_interval": "5.000000", 2023-01-25T05:01:17.155 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_xattr_value_size": "0", 2023-01-25T05:01:17.156 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-01-25T05:01:17.156 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-01-25T05:01:17.156 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-01-25T05:01:17.156 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_merge_threshold": "-10", 2023-01-25T05:01:17.156 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_min_sync_interval": "0.010000", 2023-01-25T05:01:17.157 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_odsync_write": "false", 2023-01-25T05:01:17.157 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_omap_backend": "rocksdb", 2023-01-25T05:01:17.157 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_omap_backend_path": "", 2023-01-25T05:01:17.157 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_omap_header_cache_size": "1024", 2023-01-25T05:01:17.157 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_ondisk_finisher_threads": "1", 2023-01-25T05:01:17.158 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-01-25T05:01:17.158 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_op_thread_timeout": "60", 2023-01-25T05:01:17.158 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_op_threads": "2", 2023-01-25T05:01:17.158 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_punch_hole": "false", 2023-01-25T05:01:17.158 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-01-25T05:01:17.159 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-01-25T05:01:17.159 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-01-25T05:01:17.159 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-01-25T05:01:17.159 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-01-25T05:01:17.159 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_queue_max_bytes": "104857600", 2023-01-25T05:01:17.160 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-01-25T05:01:17.160 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-01-25T05:01:17.160 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-01-25T05:01:17.160 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_queue_max_ops": "50", 2023-01-25T05:01:17.160 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-01-25T05:01:17.161 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_seek_data_hole": "false", 2023-01-25T05:01:17.161 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_sloppy_crc": "false", 2023-01-25T05:01:17.161 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-01-25T05:01:17.161 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_splice": "false", 2023-01-25T05:01:17.161 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_split_multiple": "2", 2023-01-25T05:01:17.162 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_split_rand_factor": "20", 2023-01-25T05:01:17.162 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_update_to": "1000", 2023-01-25T05:01:17.162 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-01-25T05:01:17.162 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-01-25T05:01:17.162 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-01-25T05:01:17.163 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-01-25T05:01:17.163 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-01-25T05:01:17.163 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-01-25T05:01:17.163 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_enable": "true", 2023-01-25T05:01:17.163 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-01-25T05:01:17.164 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-01-25T05:01:17.164 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-01-25T05:01:17.164 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-01-25T05:01:17.164 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-01-25T05:01:17.164 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-01-25T05:01:17.165 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_xfs_extsize": "false", 2023-01-25T05:01:17.165 INFO:teuthology.orchestra.run.smithi135.stdout: "filestore_zfs_snap": "false", 2023-01-25T05:01:17.165 INFO:teuthology.orchestra.run.smithi135.stdout: "fio_dir": "/tmp/fio", 2023-01-25T05:01:17.165 INFO:teuthology.orchestra.run.smithi135.stdout: "fsid": "8729840b-5c6f-4675-8457-dfb384ce25b3", 2023-01-25T05:01:17.165 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_allow_other": "true", 2023-01-25T05:01:17.166 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_atomic_o_trunc": "true", 2023-01-25T05:01:17.166 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_big_writes": "true", 2023-01-25T05:01:17.166 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_debug": "false", 2023-01-25T05:01:17.166 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_default_permissions": "false", 2023-01-25T05:01:17.166 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_disable_pagecache": "false", 2023-01-25T05:01:17.167 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_max_write": "0", 2023-01-25T05:01:17.167 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_multithreaded": "true", 2023-01-25T05:01:17.167 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_require_active_mds": "true", 2023-01-25T05:01:17.167 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_set_user_groups": "true", 2023-01-25T05:01:17.167 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_splice_move": "true", 2023-01-25T05:01:17.168 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_splice_read": "true", 2023-01-25T05:01:17.168 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_splice_write": "true", 2023-01-25T05:01:17.168 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_syncfs_on_mksnap": "true", 2023-01-25T05:01:17.168 INFO:teuthology.orchestra.run.smithi135.stdout: "fuse_use_invalidate_cb": "true", 2023-01-25T05:01:17.168 INFO:teuthology.orchestra.run.smithi135.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-01-25T05:01:17.169 INFO:teuthology.orchestra.run.smithi135.stdout: "gss_target_name": "ceph", 2023-01-25T05:01:17.169 INFO:teuthology.orchestra.run.smithi135.stdout: "heartbeat_file": "", 2023-01-25T05:01:17.169 INFO:teuthology.orchestra.run.smithi135.stdout: "heartbeat_inject_failure": "0", 2023-01-25T05:01:17.169 INFO:teuthology.orchestra.run.smithi135.stdout: "heartbeat_interval": "5", 2023-01-25T05:01:17.169 INFO:teuthology.orchestra.run.smithi135.stdout: "host": "smithi135", 2023-01-25T05:01:17.170 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-01-25T05:01:17.170 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-01-25T05:01:17.170 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_max_size": "1073741824", 2023-01-25T05:01:17.170 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-01-25T05:01:17.170 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-01-25T05:01:17.170 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-01-25T05:01:17.171 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-01-25T05:01:17.171 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-01-25T05:01:17.171 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-01-25T05:01:17.171 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-01-25T05:01:17.171 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-01-25T05:01:17.172 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-01-25T05:01:17.172 INFO:teuthology.orchestra.run.smithi135.stdout: "immutable_object_cache_watermark": "0.900000", 2023-01-25T05:01:17.172 INFO:teuthology.orchestra.run.smithi135.stdout: "inject_early_sigterm": "false", 2023-01-25T05:01:17.172 INFO:teuthology.orchestra.run.smithi135.stdout: "jaeger_tracing_enable": "false", 2023-01-25T05:01:17.172 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_aio": "true", 2023-01-25T05:01:17.173 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_align_min_size": "65536", 2023-01-25T05:01:17.173 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_block_align": "true", 2023-01-25T05:01:17.173 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_block_size": "4096", 2023-01-25T05:01:17.173 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_dio": "true", 2023-01-25T05:01:17.173 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_discard": "false", 2023-01-25T05:01:17.174 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_force_aio": "false", 2023-01-25T05:01:17.174 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_ignore_corruption": "false", 2023-01-25T05:01:17.174 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_max_write_bytes": "10485760", 2023-01-25T05:01:17.174 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_max_write_entries": "100", 2023-01-25T05:01:17.174 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_replay_from": "0", 2023-01-25T05:01:17.175 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_throttle_high_multiple": "0.000000", 2023-01-25T05:01:17.175 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-01-25T05:01:17.175 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-01-25T05:01:17.175 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_throttle_max_multiple": "0.000000", 2023-01-25T05:01:17.175 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_write_header_frequency": "0", 2023-01-25T05:01:17.176 INFO:teuthology.orchestra.run.smithi135.stdout: "journal_zero_on_create": "false", 2023-01-25T05:01:17.176 INFO:teuthology.orchestra.run.smithi135.stdout: "journaler_prefetch_periods": "10", 2023-01-25T05:01:17.176 INFO:teuthology.orchestra.run.smithi135.stdout: "journaler_prezero_periods": "5", 2023-01-25T05:01:17.176 INFO:teuthology.orchestra.run.smithi135.stdout: "journaler_write_head_interval": "15", 2023-01-25T05:01:17.176 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "", 2023-01-25T05:01:17.177 INFO:teuthology.orchestra.run.smithi135.stdout: "keyfile": "", 2023-01-25T05:01:17.177 INFO:teuthology.orchestra.run.smithi135.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-01-25T05:01:17.177 INFO:teuthology.orchestra.run.smithi135.stdout: "kstore_backend": "rocksdb", 2023-01-25T05:01:17.177 INFO:teuthology.orchestra.run.smithi135.stdout: "kstore_default_stripe_size": "65536", 2023-01-25T05:01:17.177 INFO:teuthology.orchestra.run.smithi135.stdout: "kstore_fsck_on_mount": "false", 2023-01-25T05:01:17.178 INFO:teuthology.orchestra.run.smithi135.stdout: "kstore_fsck_on_mount_deep": "true", 2023-01-25T05:01:17.178 INFO:teuthology.orchestra.run.smithi135.stdout: "kstore_max_bytes": "67108864", 2023-01-25T05:01:17.178 INFO:teuthology.orchestra.run.smithi135.stdout: "kstore_max_ops": "512", 2023-01-25T05:01:17.178 INFO:teuthology.orchestra.run.smithi135.stdout: "kstore_nid_prealloc": "1024", 2023-01-25T05:01:17.178 INFO:teuthology.orchestra.run.smithi135.stdout: "kstore_onode_map_size": "1024", 2023-01-25T05:01:17.178 INFO:teuthology.orchestra.run.smithi135.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-01-25T05:01:17.179 INFO:teuthology.orchestra.run.smithi135.stdout: "kstore_sync_submit_transaction": "false", 2023-01-25T05:01:17.179 INFO:teuthology.orchestra.run.smithi135.stdout: "kstore_sync_transaction": "false", 2023-01-25T05:01:17.179 INFO:teuthology.orchestra.run.smithi135.stdout: "leveldb_block_size": "65536", 2023-01-25T05:01:17.179 INFO:teuthology.orchestra.run.smithi135.stdout: "leveldb_bloom_size": "0", 2023-01-25T05:01:17.179 INFO:teuthology.orchestra.run.smithi135.stdout: "leveldb_cache_size": "536870912", 2023-01-25T05:01:17.180 INFO:teuthology.orchestra.run.smithi135.stdout: "leveldb_compact_on_mount": "false", 2023-01-25T05:01:17.180 INFO:teuthology.orchestra.run.smithi135.stdout: "leveldb_compression": "false", 2023-01-25T05:01:17.180 INFO:teuthology.orchestra.run.smithi135.stdout: "leveldb_log": "", 2023-01-25T05:01:17.180 INFO:teuthology.orchestra.run.smithi135.stdout: "leveldb_log_to_ceph_log": "true", 2023-01-25T05:01:17.180 INFO:teuthology.orchestra.run.smithi135.stdout: "leveldb_max_open_files": "0", 2023-01-25T05:01:17.181 INFO:teuthology.orchestra.run.smithi135.stdout: "leveldb_paranoid": "false", 2023-01-25T05:01:17.181 INFO:teuthology.orchestra.run.smithi135.stdout: "leveldb_write_buffer_size": "33554432", 2023-01-25T05:01:17.181 INFO:teuthology.orchestra.run.smithi135.stdout: "librados_thread_count": "2", 2023-01-25T05:01:17.181 INFO:teuthology.orchestra.run.smithi135.stdout: "lockdep": "false", 2023-01-25T05:01:17.181 INFO:teuthology.orchestra.run.smithi135.stdout: "lockdep_force_backtrace": "false", 2023-01-25T05:01:17.182 INFO:teuthology.orchestra.run.smithi135.stdout: "log_coarse_timestamps": "true", 2023-01-25T05:01:17.182 INFO:teuthology.orchestra.run.smithi135.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-01-25T05:01:17.182 INFO:teuthology.orchestra.run.smithi135.stdout: "log_flush_on_exit": "false", 2023-01-25T05:01:17.182 INFO:teuthology.orchestra.run.smithi135.stdout: "log_graylog_host": "127.0.0.1", 2023-01-25T05:01:17.182 INFO:teuthology.orchestra.run.smithi135.stdout: "log_graylog_port": "12201", 2023-01-25T05:01:17.182 INFO:teuthology.orchestra.run.smithi135.stdout: "log_max_new": "1000", 2023-01-25T05:01:17.183 INFO:teuthology.orchestra.run.smithi135.stdout: "log_max_recent": "10000", 2023-01-25T05:01:17.183 INFO:teuthology.orchestra.run.smithi135.stdout: "log_stderr_prefix": "", 2023-01-25T05:01:17.183 INFO:teuthology.orchestra.run.smithi135.stdout: "log_stop_at_utilization": "0.970000", 2023-01-25T05:01:17.183 INFO:teuthology.orchestra.run.smithi135.stdout: "log_to_file": "true", 2023-01-25T05:01:17.183 INFO:teuthology.orchestra.run.smithi135.stdout: "log_to_graylog": "false", 2023-01-25T05:01:17.184 INFO:teuthology.orchestra.run.smithi135.stdout: "log_to_journald": "false", 2023-01-25T05:01:17.184 INFO:teuthology.orchestra.run.smithi135.stdout: "log_to_stderr": "false", 2023-01-25T05:01:17.184 INFO:teuthology.orchestra.run.smithi135.stdout: "log_to_syslog": "false", 2023-01-25T05:01:17.184 INFO:teuthology.orchestra.run.smithi135.stdout: "max_rotating_auth_attempts": "10", 2023-01-25T05:01:17.184 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_action_on_write_error": "1", 2023-01-25T05:01:17.185 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_alternate_name_max": "8192", 2023-01-25T05:01:17.185 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_asio_thread_count": "2", 2023-01-25T05:01:17.185 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_export_pin": "true", 2023-01-25T05:01:17.185 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_fragment_dirs": "true", 2023-01-25T05:01:17.185 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-01-25T05:01:17.186 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_fragment_interval": "5", 2023-01-25T05:01:17.186 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_fragment_size_max": "100000", 2023-01-25T05:01:17.186 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_idle_threshold": "0.000000", 2023-01-25T05:01:17.186 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_interval": "10", 2023-01-25T05:01:17.187 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_max": "-1", 2023-01-25T05:01:17.187 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_max_until": "-1", 2023-01-25T05:01:17.187 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_merge_size": "50", 2023-01-25T05:01:17.187 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_midchunk": "0.300000", 2023-01-25T05:01:17.188 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_min_rebalance": "0.100000", 2023-01-25T05:01:17.188 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_min_start": "0.200000", 2023-01-25T05:01:17.188 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_minchunk": "0.001000", 2023-01-25T05:01:17.188 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_mode": "0", 2023-01-25T05:01:17.188 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_need_max": "1.200000", 2023-01-25T05:01:17.189 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_need_min": "0.800000", 2023-01-25T05:01:17.189 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-01-25T05:01:17.189 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_sample_interval": "3.000000", 2023-01-25T05:01:17.189 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_split_bits": "3", 2023-01-25T05:01:17.189 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_split_rd": "25000.000000", 2023-01-25T05:01:17.190 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_split_size": "10000", 2023-01-25T05:01:17.190 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_split_wr": "10000.000000", 2023-01-25T05:01:17.190 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_target_decay": "10.000000", 2023-01-25T05:01:17.190 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-01-25T05:01:17.191 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_beacon_grace": "15.000000", 2023-01-25T05:01:17.191 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_beacon_interval": "4.000000", 2023-01-25T05:01:17.191 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_beacon_mon_down_grace": "60", 2023-01-25T05:01:17.191 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache_memory_limit": "4294967296", 2023-01-25T05:01:17.191 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache_mid": "0.700000", 2023-01-25T05:01:17.192 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache_release_free_interval": "10", 2023-01-25T05:01:17.192 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache_reservation": "0.050000", 2023-01-25T05:01:17.192 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-01-25T05:01:17.192 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache_trim_interval": "1", 2023-01-25T05:01:17.192 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache_trim_threshold": "262144", 2023-01-25T05:01:17.193 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-01-25T05:01:17.193 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-01-25T05:01:17.193 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_client_delegate_inos_pct": "50", 2023-01-25T05:01:17.193 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_client_prealloc_inos": "1000", 2023-01-25T05:01:17.193 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-01-25T05:01:17.194 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_connect_bootstrapping": "false", 2023-01-25T05:01:17.194 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_damage_table_max_entries": "10000", 2023-01-25T05:01:17.194 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-01-25T05:01:17.194 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_debug_auth_pins": "false", 2023-01-25T05:01:17.194 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_debug_frag": "false", 2023-01-25T05:01:17.195 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_debug_scatterstat": "false", 2023-01-25T05:01:17.195 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_debug_subtrees": "false", 2023-01-25T05:01:17.195 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_decay_halflife": "5.000000", 2023-01-25T05:01:17.195 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_default_dir_hash": "2", 2023-01-25T05:01:17.195 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_defer_session_stale": "true", 2023-01-25T05:01:17.196 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_deny_all_reconnect": "false", 2023-01-25T05:01:17.196 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_dir_keys_per_op": "16384", 2023-01-25T05:01:17.196 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_dir_max_commit_size": "10", 2023-01-25T05:01:17.196 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_dir_max_entries": "0", 2023-01-25T05:01:17.196 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_dir_prefetch": "true", 2023-01-25T05:01:17.197 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_dirstat_min_interval": "1.000000", 2023-01-25T05:01:17.197 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_dump_cache_after_rejoin": "false", 2023-01-25T05:01:17.197 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_dump_cache_on_map": "false", 2023-01-25T05:01:17.197 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_dump_cache_threshold_file": "0", 2023-01-25T05:01:17.197 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-01-25T05:01:17.198 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_early_reply": "true", 2023-01-25T05:01:17.198 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_enable_op_tracker": "true", 2023-01-25T05:01:17.198 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_enforce_unique_name": "true", 2023-01-25T05:01:17.198 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_export_ephemeral_distributed": "true", 2023-01-25T05:01:17.198 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-01-25T05:01:17.198 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_export_ephemeral_random": "true", 2023-01-25T05:01:17.199 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-01-25T05:01:17.199 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_extraordinary_events_dump_interval": "0", 2023-01-25T05:01:17.199 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_forward_all_requests_to_auth": "false", 2023-01-25T05:01:17.199 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-01-25T05:01:17.199 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_fscrypt_last_block_max_size": "4096", 2023-01-25T05:01:17.200 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-01-25T05:01:17.200 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_health_cache_threshold": "1.500000", 2023-01-25T05:01:17.200 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_health_summarize_threshold": "10", 2023-01-25T05:01:17.200 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_heartbeat_grace": "15.000000", 2023-01-25T05:01:17.200 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_heartbeat_reset_grace": "1000", 2023-01-25T05:01:17.201 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_inject_health_dummy": "false", 2023-01-25T05:01:17.201 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_inject_migrator_session_race": "false", 2023-01-25T05:01:17.201 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-01-25T05:01:17.201 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_join_fs": "", 2023-01-25T05:01:17.201 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_journal_format": "1", 2023-01-25T05:01:17.202 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_kill_create_at": "0", 2023-01-25T05:01:17.202 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_kill_export_at": "0", 2023-01-25T05:01:17.202 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_kill_import_at": "0", 2023-01-25T05:01:17.202 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_kill_journal_at": "0", 2023-01-25T05:01:17.202 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_kill_journal_expire_at": "0", 2023-01-25T05:01:17.202 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_kill_journal_replay_at": "0", 2023-01-25T05:01:17.203 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_kill_link_at": "0", 2023-01-25T05:01:17.203 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_kill_mdstable_at": "0", 2023-01-25T05:01:17.203 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_kill_openc_at": "0", 2023-01-25T05:01:17.203 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_kill_rename_at": "0", 2023-01-25T05:01:17.203 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log_events_per_segment": "1024", 2023-01-25T05:01:17.204 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log_max_events": "-1", 2023-01-25T05:01:17.204 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log_max_segments": "128", 2023-01-25T05:01:17.204 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log_pause": "false", 2023-01-25T05:01:17.204 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log_segment_size": "0", 2023-01-25T05:01:17.204 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log_skip_corrupt_events": "false", 2023-01-25T05:01:17.204 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log_warn_factor": "2.000000", 2023-01-25T05:01:17.205 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_caps_per_client": "1048576", 2023-01-25T05:01:17.205 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_completed_flushes": "100000", 2023-01-25T05:01:17.205 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_completed_requests": "100000", 2023-01-25T05:01:17.205 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_export_size": "20971520", 2023-01-25T05:01:17.205 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_file_recover": "32", 2023-01-25T05:01:17.206 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_purge_files": "64", 2023-01-25T05:01:17.206 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_purge_ops": "8192", 2023-01-25T05:01:17.206 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-01-25T05:01:17.206 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_retries_on_remount_failure": "5", 2023-01-25T05:01:17.206 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-01-25T05:01:17.207 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_snaps_per_dir": "100", 2023-01-25T05:01:17.207 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_max_xattr_pairs_size": "65536", 2023-01-25T05:01:17.207 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_metrics_update_interval": "2", 2023-01-25T05:01:17.207 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_min_caps_per_client": "100", 2023-01-25T05:01:17.207 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_min_caps_working_set": "10000", 2023-01-25T05:01:17.208 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-01-25T05:01:17.208 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_numa_node": "-1", 2023-01-25T05:01:17.208 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-01-25T05:01:17.208 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_op_complaint_time": "30.000000", 2023-01-25T05:01:17.208 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_op_history_duration": "600", 2023-01-25T05:01:17.209 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_op_history_size": "20", 2023-01-25T05:01:17.209 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_op_log_threshold": "5", 2023-01-25T05:01:17.209 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_ping_grace": "15", 2023-01-25T05:01:17.209 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_ping_interval": "5", 2023-01-25T05:01:17.209 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-01-25T05:01:17.210 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-01-25T05:01:17.210 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_recall_max_caps": "30000", 2023-01-25T05:01:17.210 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-01-25T05:01:17.210 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_recall_max_decay_threshold": "131072", 2023-01-25T05:01:17.210 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-01-25T05:01:17.210 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_recall_warning_threshold": "262144", 2023-01-25T05:01:17.211 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_reconnect_timeout": "45.000000", 2023-01-25T05:01:17.211 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_replay_interval": "1.000000", 2023-01-25T05:01:17.211 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-01-25T05:01:17.211 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-01-25T05:01:17.212 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_root_ino_gid": "0", 2023-01-25T05:01:17.212 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_root_ino_uid": "0", 2023-01-25T05:01:17.212 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-01-25T05:01:17.212 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_session_blocklist_on_evict": "true", 2023-01-25T05:01:17.212 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_session_blocklist_on_timeout": "true", 2023-01-25T05:01:17.213 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-01-25T05:01:17.213 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-01-25T05:01:17.213 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2023-01-25T05:01:17.213 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_session_cap_acquisition_throttle": "500000", 2023-01-25T05:01:17.213 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-01-25T05:01:17.214 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-01-25T05:01:17.214 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_shutdown_check": "0", 2023-01-25T05:01:17.214 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_skip_ino": "0", 2023-01-25T05:01:17.214 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sleep_rank_change": "0.000000", 2023-01-25T05:01:17.214 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_snap_max_uid": "4294967294", 2023-01-25T05:01:17.215 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_snap_min_uid": "0", 2023-01-25T05:01:17.215 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_snap_rstat": "false", 2023-01-25T05:01:17.215 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_standby_replay_damaged": "false", 2023-01-25T05:01:17.215 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_symlink_recovery": "true", 2023-01-25T05:01:17.215 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_task_status_update_interval": "2.000000", 2023-01-25T05:01:17.216 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_thrash_exports": "0", 2023-01-25T05:01:17.216 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_thrash_fragments": "0", 2023-01-25T05:01:17.216 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_tick_interval": "5.000000", 2023-01-25T05:01:17.216 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_valgrind_exit": "false", 2023-01-25T05:01:17.216 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_verify_backtrace": "1", 2023-01-25T05:01:17.217 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_verify_scatter": "false", 2023-01-25T05:01:17.217 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_wipe_ino_prealloc": "false", 2023-01-25T05:01:17.217 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_wipe_sessions": "false", 2023-01-25T05:01:17.217 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool_debug": "false", 2023-01-25T05:01:17.217 INFO:teuthology.orchestra.run.smithi135.stdout: "memstore_debug_omit_block_device_write": "false", 2023-01-25T05:01:17.217 INFO:teuthology.orchestra.run.smithi135.stdout: "memstore_device_bytes": "1073741824", 2023-01-25T05:01:17.218 INFO:teuthology.orchestra.run.smithi135.stdout: "memstore_page_set": "false", 2023-01-25T05:01:17.218 INFO:teuthology.orchestra.run.smithi135.stdout: "memstore_page_size": "65536", 2023-01-25T05:01:17.218 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_client_bytes": "134217728", 2023-01-25T05:01:17.218 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_client_messages": "512", 2023-01-25T05:01:17.218 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-01-25T05:01:17.219 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_connect_retry_interval": "1.000000", 2023-01-25T05:01:17.219 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-01-25T05:01:17.219 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-01-25T05:01:17.219 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_disabled_modules": "", 2023-01-25T05:01:17.219 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-01-25T05:01:17.220 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_max_pg_num_change": "128", 2023-01-25T05:01:17.220 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_mds_bytes": "134217728", 2023-01-25T05:01:17.220 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_mds_messages": "128", 2023-01-25T05:01:17.220 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-01-25T05:01:17.220 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_mon_bytes": "134217728", 2023-01-25T05:01:17.221 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_mon_messages": "128", 2023-01-25T05:01:17.221 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_osd_bytes": "536870912", 2023-01-25T05:01:17.221 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_osd_messages": "8192", 2023-01-25T05:01:17.221 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_pool": "true", 2023-01-25T05:01:17.221 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_service_beacon_grace": "60.000000", 2023-01-25T05:01:17.222 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_standby_modules": "true", 2023-01-25T05:01:17.222 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_stats_period": "5", 2023-01-25T05:01:17.222 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_stats_threshold": "5", 2023-01-25T05:01:17.222 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_tick_period": "2", 2023-01-25T05:01:17.222 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-01-25T05:01:17.222 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_accept_timeout_factor": "2.000000", 2023-01-25T05:01:17.223 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_allow_pool_delete": "true", 2023-01-25T05:01:17.223 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_allow_pool_size_one": "true", 2023-01-25T05:01:17.223 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_auth_validate_all_caps": "true", 2023-01-25T05:01:17.223 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-01-25T05:01:17.223 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-01-25T05:01:17.224 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_client_bytes": "104857600", 2023-01-25T05:01:17.224 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_client_directed_command_retry": "2", 2023-01-25T05:01:17.224 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_client_hunt_interval": "3.000000", 2023-01-25T05:01:17.224 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-01-25T05:01:17.224 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-01-25T05:01:17.225 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-01-25T05:01:17.225 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_client_hunt_parallel": "3", 2023-01-25T05:01:17.225 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_client_log_interval": "1.000000", 2023-01-25T05:01:17.225 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-01-25T05:01:17.225 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_client_ping_interval": "10.000000", 2023-01-25T05:01:17.226 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_client_ping_timeout": "30.000000", 2023-01-25T05:01:17.226 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_clock_drift_allowed": "1.000000", 2023-01-25T05:01:17.226 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-01-25T05:01:17.226 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-01-25T05:01:17.226 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cluster_log_file_level": "debug", 2023-01-25T05:01:17.227 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cluster_log_to_file": "true", 2023-01-25T05:01:17.227 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cluster_log_to_graylog": "false", 2023-01-25T05:01:17.227 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-01-25T05:01:17.227 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-01-25T05:01:17.228 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cluster_log_to_journald": "false", 2023-01-25T05:01:17.228 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cluster_log_to_stderr": "false", 2023-01-25T05:01:17.228 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-01-25T05:01:17.228 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-01-25T05:01:17.228 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-01-25T05:01:17.229 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_compact_on_bootstrap": "false", 2023-01-25T05:01:17.229 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_compact_on_start": "false", 2023-01-25T05:01:17.229 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_compact_on_trim": "true", 2023-01-25T05:01:17.229 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_con_tracker_persist_interval": "10", 2023-01-25T05:01:17.229 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_con_tracker_score_halflife": "43200", 2023-01-25T05:01:17.230 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_config_key_max_entry_size": "65536", 2023-01-25T05:01:17.230 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_cpu_threads": "4", 2023-01-25T05:01:17.230 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_crush_min_required_version": "hammer", 2023-01-25T05:01:17.230 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_daemon_bytes": "419430400", 2023-01-25T05:01:17.230 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-01-25T05:01:17.231 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_data_avail_crit": "5", 2023-01-25T05:01:17.231 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_data_avail_warn": "5", 2023-01-25T05:01:17.231 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_data_size_warn": "16106127360", 2023-01-25T05:01:17.231 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_debug_block_osdmap_trim": "false", 2023-01-25T05:01:17.231 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-01-25T05:01:17.232 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_debug_dump_json": "false", 2023-01-25T05:01:17.232 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-01-25T05:01:17.232 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_debug_dump_transactions": "false", 2023-01-25T05:01:17.232 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_debug_extra_checks": "false", 2023-01-25T05:01:17.232 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-01-25T05:01:17.233 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-01-25T05:01:17.233 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_debug_no_require_quincy": "false", 2023-01-25T05:01:17.233 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_debug_no_require_reef": "false", 2023-01-25T05:01:17.233 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-01-25T05:01:17.233 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_delta_reset_interval": "10.000000", 2023-01-25T05:01:17.234 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_dns_srv_name": "ceph-mon", 2023-01-25T05:01:17.234 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_down_mkfs_grace": "120", 2023-01-25T05:01:17.234 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_election_default_strategy": "1", 2023-01-25T05:01:17.234 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_election_timeout": "5.000000", 2023-01-25T05:01:17.234 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-01-25T05:01:17.234 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_elector_ping_divisor": "2", 2023-01-25T05:01:17.235 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_elector_ping_timeout": "2.000000", 2023-01-25T05:01:17.235 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_enable_op_tracker": "true", 2023-01-25T05:01:17.235 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_fake_pool_delete": "false", 2023-01-25T05:01:17.235 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_force_quorum_join": "false", 2023-01-25T05:01:17.235 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_globalid_prealloc": "10000", 2023-01-25T05:01:17.236 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_health_detail_to_clog": "false", 2023-01-25T05:01:17.236 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_health_log_update_period": "5", 2023-01-25T05:01:17.236 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_health_max_detail": "50", 2023-01-25T05:01:17.236 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_health_to_clog": "true", 2023-01-25T05:01:17.236 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_health_to_clog_interval": "600", 2023-01-25T05:01:17.237 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-01-25T05:01:17.237 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_host": "172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790],172.21.15.178", 2023-01-25T05:01:17.237 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_host_override": "", 2023-01-25T05:01:17.237 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_initial_members": "", 2023-01-25T05:01:17.237 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-01-25T05:01:17.238 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-01-25T05:01:17.238 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-01-25T05:01:17.238 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-01-25T05:01:17.238 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_keyvaluedb": "rocksdb", 2023-01-25T05:01:17.238 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_lease": "5.000000", 2023-01-25T05:01:17.239 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-01-25T05:01:17.239 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-01-25T05:01:17.239 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_log_full_interval": "50", 2023-01-25T05:01:17.239 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_log_max": "10000", 2023-01-25T05:01:17.239 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_log_max_summary": "50", 2023-01-25T05:01:17.240 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_max_log_entries_per_event": "4096", 2023-01-25T05:01:17.240 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_max_log_epochs": "500", 2023-01-25T05:01:17.240 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_max_mdsmap_epochs": "500", 2023-01-25T05:01:17.240 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_max_mgrmap_epochs": "500", 2023-01-25T05:01:17.241 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_max_osd": "10000", 2023-01-25T05:01:17.241 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_max_pg_per_osd": "10000", 2023-01-25T05:01:17.241 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_max_pool_pg_num": "65536", 2023-01-25T05:01:17.241 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-01-25T05:01:17.241 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-01-25T05:01:17.242 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_mds_force_trim_to": "0", 2023-01-25T05:01:17.242 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_mds_skip_sanity": "false", 2023-01-25T05:01:17.242 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_memory_autotune": "true", 2023-01-25T05:01:17.242 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_memory_target": "2147483648", 2023-01-25T05:01:17.242 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_mgr_beacon_grace": "30", 2023-01-25T05:01:17.243 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-01-25T05:01:17.243 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_mgr_digest_period": "5", 2023-01-25T05:01:17.243 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_mgr_inactive_grace": "60", 2023-01-25T05:01:17.243 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_mgr_mkfs_grace": "240", 2023-01-25T05:01:17.243 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-01-25T05:01:17.244 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_min_osdmap_epochs": "50", 2023-01-25T05:01:17.244 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_op_complaint_time": "30", 2023-01-25T05:01:17.244 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_op_history_duration": "600", 2023-01-25T05:01:17.244 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_op_history_size": "20", 2023-01-25T05:01:17.244 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_op_history_slow_op_size": "20", 2023-01-25T05:01:17.245 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_op_history_slow_op_threshold": "10", 2023-01-25T05:01:17.245 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_op_log_threshold": "5", 2023-01-25T05:01:17.245 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-01-25T05:01:17.245 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-01-25T05:01:17.245 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-01-25T05:01:17.246 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_auto_mark_in": "false", 2023-01-25T05:01:17.246 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_auto_mark_new_in": "true", 2023-01-25T05:01:17.246 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-01-25T05:01:17.246 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-01-25T05:01:17.246 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_cache_size": "500", 2023-01-25T05:01:17.247 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_cache_size_min": "134217728", 2023-01-25T05:01:17.247 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_crush_smoke_test": "true", 2023-01-25T05:01:17.247 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_destroyed_out_interval": "600", 2023-01-25T05:01:17.247 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_down_out_interval": "600", 2023-01-25T05:01:17.247 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-01-25T05:01:17.248 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-01-25T05:01:17.248 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_force_trim_to": "0", 2023-01-25T05:01:17.248 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_full_ratio": "0.950000", 2023-01-25T05:01:17.248 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-01-25T05:01:17.248 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_laggy_halflife": "3600", 2023-01-25T05:01:17.249 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_laggy_max_interval": "300", 2023-01-25T05:01:17.249 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_laggy_weight": "0.300000", 2023-01-25T05:01:17.249 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-01-25T05:01:17.249 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_max_creating_pgs": "1024", 2023-01-25T05:01:17.250 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_max_initial_pgs": "1024", 2023-01-25T05:01:17.250 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_min_down_reporters": "2", 2023-01-25T05:01:17.250 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-01-25T05:01:17.250 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-01-25T05:01:17.250 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-01-25T05:01:17.251 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_prime_pg_temp": "true", 2023-01-25T05:01:17.251 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-01-25T05:01:17.251 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-01-25T05:01:17.251 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_report_timeout": "900", 2023-01-25T05:01:17.252 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-01-25T05:01:17.252 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-01-25T05:01:17.252 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_warn_num_repaired": "10", 2023-01-25T05:01:17.252 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osd_warn_op_age": "32.000000", 2023-01-25T05:01:17.253 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-01-25T05:01:17.253 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osdmap_full_prune_interval": "2", 2023-01-25T05:01:17.253 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osdmap_full_prune_min": "15", 2023-01-25T05:01:17.253 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_osdmap_full_prune_txsize": "2", 2023-01-25T05:01:17.253 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-01-25T05:01:17.254 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_pg_stuck_threshold": "60", 2023-01-25T05:01:17.254 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-01-25T05:01:17.254 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_pg_warn_min_objects": "10000", 2023-01-25T05:01:17.254 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_pg_warn_min_per_osd": "0", 2023-01-25T05:01:17.255 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-01-25T05:01:17.255 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_pool_quota_crit_threshold": "0", 2023-01-25T05:01:17.255 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_pool_quota_warn_threshold": "0", 2023-01-25T05:01:17.255 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_probe_timeout": "2.000000", 2023-01-25T05:01:17.255 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_reweight_max_change": "0.050000", 2023-01-25T05:01:17.256 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_reweight_max_osds": "4", 2023-01-25T05:01:17.256 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-01-25T05:01:17.256 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-01-25T05:01:17.256 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-01-25T05:01:17.257 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-01-25T05:01:17.257 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-01-25T05:01:17.257 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_scrub_interval": "300", 2023-01-25T05:01:17.257 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_scrub_max_keys": "100", 2023-01-25T05:01:17.258 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_scrub_timeout": "300", 2023-01-25T05:01:17.258 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_session_timeout": "300", 2023-01-25T05:01:17.258 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_smart_report_timeout": "5", 2023-01-25T05:01:17.258 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_stat_smooth_intervals": "6", 2023-01-25T05:01:17.258 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-01-25T05:01:17.259 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_stretch_pool_min_size": "2", 2023-01-25T05:01:17.259 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_stretch_pool_size": "4", 2023-01-25T05:01:17.259 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-01-25T05:01:17.259 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_subscribe_interval": "86400.000000", 2023-01-25T05:01:17.259 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_sync_debug": "false", 2023-01-25T05:01:17.260 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_sync_max_payload_keys": "2000", 2023-01-25T05:01:17.260 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_sync_max_payload_size": "1048576", 2023-01-25T05:01:17.260 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_sync_provider_kill_at": "0", 2023-01-25T05:01:17.260 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_sync_requester_kill_at": "0", 2023-01-25T05:01:17.260 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_sync_timeout": "60.000000", 2023-01-25T05:01:17.261 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_target_pg_per_osd": "100", 2023-01-25T05:01:17.261 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_tick_interval": "5", 2023-01-25T05:01:17.261 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_timecheck_interval": "300.000000", 2023-01-25T05:01:17.261 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-01-25T05:01:17.261 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_use_min_delay_socket": "false", 2023-01-25T05:01:17.262 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_older_version_delay": "604800", 2023-01-25T05:01:17.262 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-01-25T05:01:17.262 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-01-25T05:01:17.262 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-01-25T05:01:17.262 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_filestore_osds": "false", 2023-01-25T05:01:17.263 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-01-25T05:01:17.263 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-01-25T05:01:17.263 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-01-25T05:01:17.263 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_misplaced": "false", 2023-01-25T05:01:17.263 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-01-25T05:01:17.264 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_older_version": "true", 2023-01-25T05:01:17.264 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-01-25T05:01:17.264 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_pool_no_app": "true", 2023-01-25T05:01:17.264 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-01-25T05:01:17.264 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-01-25T05:01:17.265 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-01-25T05:01:17.265 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-01-25T05:01:17.265 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_on_too_few_osds": "false", 2023-01-25T05:01:17.265 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-01-25T05:01:17.265 INFO:teuthology.orchestra.run.smithi135.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-01-25T05:01:17.266 INFO:teuthology.orchestra.run.smithi135.stdout: "monmap": "", 2023-01-25T05:01:17.266 INFO:teuthology.orchestra.run.smithi135.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2023-01-25T05:01:17.266 INFO:teuthology.orchestra.run.smithi135.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-01-25T05:01:17.266 INFO:teuthology.orchestra.run.smithi135.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2023-01-25T05:01:17.266 INFO:teuthology.orchestra.run.smithi135.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2023-01-25T05:01:17.267 INFO:teuthology.orchestra.run.smithi135.stdout: "motr_tracing_enabled": "false", 2023-01-25T05:01:17.267 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_op_threads": "3", 2023-01-25T05:01:17.267 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_buffer_size": "131072", 2023-01-25T05:01:17.267 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_cm": "false", 2023-01-25T05:01:17.267 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_device_name": "", 2023-01-25T05:01:17.268 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_dscp": "96", 2023-01-25T05:01:17.268 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_enable_hugepage": "false", 2023-01-25T05:01:17.268 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_gid_idx": "0", 2023-01-25T05:01:17.268 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_local_gid": "", 2023-01-25T05:01:17.269 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_polling_us": "1000", 2023-01-25T05:01:17.269 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_port_num": "1", 2023-01-25T05:01:17.269 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_receive_buffers": "32768", 2023-01-25T05:01:17.269 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_receive_queue_len": "4096", 2023-01-25T05:01:17.269 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_roce_ver": "1", 2023-01-25T05:01:17.270 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_send_buffers": "1024", 2023-01-25T05:01:17.270 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_sl": "3", 2023-01-25T05:01:17.270 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_support_srq": "true", 2023-01-25T05:01:17.270 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_rdma_type": "ib", 2023-01-25T05:01:17.270 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_async_reap_threshold": "5", 2023-01-25T05:01:17.270 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_bind_before_connect": "false", 2023-01-25T05:01:17.271 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_bind_ipv4": "true", 2023-01-25T05:01:17.271 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_bind_ipv6": "false", 2023-01-25T05:01:17.271 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_bind_msgr1": "true", 2023-01-25T05:01:17.271 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_bind_msgr2": "true", 2023-01-25T05:01:17.271 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_bind_port_max": "7568", 2023-01-25T05:01:17.272 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_bind_port_min": "6800", 2023-01-25T05:01:17.272 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_bind_prefer_ipv4": "false", 2023-01-25T05:01:17.272 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_bind_retry_count": "3", 2023-01-25T05:01:17.272 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_bind_retry_delay": "5", 2023-01-25T05:01:17.272 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_blackhole_client": "false", 2023-01-25T05:01:17.273 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_blackhole_mds": "false", 2023-01-25T05:01:17.273 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_blackhole_mgr": "false", 2023-01-25T05:01:17.273 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_blackhole_mon": "false", 2023-01-25T05:01:17.273 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_blackhole_osd": "false", 2023-01-25T05:01:17.273 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_client_mode": "crc secure", 2023-01-25T05:01:17.274 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_cluster_mode": "crc secure", 2023-01-25T05:01:17.274 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_cluster_type": "", 2023-01-25T05:01:17.274 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_compress_secure": "false", 2023-01-25T05:01:17.274 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_connection_idle_timeout": "900", 2023-01-25T05:01:17.274 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_connection_ready_timeout": "10", 2023-01-25T05:01:17.275 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_crc_data": "true", 2023-01-25T05:01:17.275 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_crc_header": "true", 2023-01-25T05:01:17.275 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_die_on_bad_msg": "false", 2023-01-25T05:01:17.275 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_die_on_bug": "true", 2023-01-25T05:01:17.275 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_die_on_old_message": "true", 2023-01-25T05:01:17.276 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_die_on_skipped_message": "false", 2023-01-25T05:01:17.276 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_die_on_unhandled_msg": "false", 2023-01-25T05:01:17.276 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dispatch_throttle_bytes": "104857600", 2023-01-25T05:01:17.276 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_coremask": "0xF", 2023-01-25T05:01:17.276 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_debug_allow_loopback": "false", 2023-01-25T05:01:17.277 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_devs_allowlist": "", 2023-01-25T05:01:17.277 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_enable_tso": "true", 2023-01-25T05:01:17.277 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2023-01-25T05:01:17.277 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_host_ipv4_addr": "", 2023-01-25T05:01:17.277 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_hugepages": "", 2023-01-25T05:01:17.277 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_hw_flow_control": "true", 2023-01-25T05:01:17.278 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2023-01-25T05:01:17.278 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_lro": "true", 2023-01-25T05:01:17.278 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_memory_channel": "4", 2023-01-25T05:01:17.278 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2023-01-25T05:01:17.278 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_pmd": "", 2023-01-25T05:01:17.279 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_port_id": "0", 2023-01-25T05:01:17.279 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2023-01-25T05:01:17.279 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dump_corrupt_message_level": "1", 2023-01-25T05:01:17.279 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_dump_on_send": "false", 2023-01-25T05:01:17.279 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_initial_backoff": "0.200000", 2023-01-25T05:01:17.280 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_inject_delay_max": "1.000000", 2023-01-25T05:01:17.280 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_inject_delay_probability": "0.000000", 2023-01-25T05:01:17.280 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_inject_delay_type": "", 2023-01-25T05:01:17.280 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_inject_internal_delays": "0.000000", 2023-01-25T05:01:17.280 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_inject_network_congestion": "0", 2023-01-25T05:01:17.281 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_inject_socket_failures": "0", 2023-01-25T05:01:17.281 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_learn_addr_from_peer": "true", 2023-01-25T05:01:17.281 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_max_accept_failures": "4", 2023-01-25T05:01:17.281 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_max_backoff": "15.000000", 2023-01-25T05:01:17.281 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_mon_client_mode": "secure crc", 2023-01-25T05:01:17.282 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_mon_cluster_mode": "secure crc", 2023-01-25T05:01:17.282 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_mon_service_mode": "secure crc", 2023-01-25T05:01:17.282 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_osd_compress_min_size": "1024", 2023-01-25T05:01:17.282 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_osd_compress_mode": "none", 2023-01-25T05:01:17.282 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2023-01-25T05:01:17.282 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2023-01-25T05:01:17.283 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_pq_min_cost": "65536", 2023-01-25T05:01:17.283 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_public_type": "", 2023-01-25T05:01:17.283 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_service_mode": "crc secure", 2023-01-25T05:01:17.283 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_tcp_listen_backlog": "512", 2023-01-25T05:01:17.283 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_tcp_nodelay": "true", 2023-01-25T05:01:17.284 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_tcp_prefetch_max_size": "4096", 2023-01-25T05:01:17.284 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_tcp_rcvbuf": "0", 2023-01-25T05:01:17.284 INFO:teuthology.orchestra.run.smithi135.stdout: "ms_type": "async", 2023-01-25T05:01:17.284 INFO:teuthology.orchestra.run.smithi135.stdout: "no_config_file": "false", 2023-01-25T05:01:17.285 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter_completion_locks_per_session": "32", 2023-01-25T05:01:17.285 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter_debug_inject_relock_delay": "false", 2023-01-25T05:01:17.285 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter_inflight_op_bytes": "104857600", 2023-01-25T05:01:17.285 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter_inflight_ops": "1024", 2023-01-25T05:01:17.285 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter_inject_no_watch_ping": "false", 2023-01-25T05:01:17.286 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter_retry_writes_after_first_reply": "false", 2023-01-25T05:01:17.286 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter_tick_interval": "5.000000", 2023-01-25T05:01:17.286 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter_timeout": "10.000000", 2023-01-25T05:01:17.286 INFO:teuthology.orchestra.run.smithi135.stdout: "objectstore_blackhole": "false", 2023-01-25T05:01:17.286 INFO:teuthology.orchestra.run.s